It's less about 'hard' and more about what you enjoy. There are many jobs out there which aren't pure back- or frontend. And within frontend, there is space for people who excel at writing idiomatic, elegant code, can cope with typesystems etc.

Branching out to backend programming won't hurt. But an alternative might be to become an expert in Typescript, or one of the more exotic technologies like ClojureScript, F# (Fable), Elm...

To learn more backend programming I'd recommend Django. It's popular, productive, and as a dynamic language very close to typescript.