I last played W3 back in June 2015 as I went overseas for a year and left my desktop behind. My game was version 1.06 before I left and now I've just patched it to 1.21. I remember most of the story events from the game and would prefer not to have to restart but I will if there may potentially be any problems caused by nearly a year's worth of patches.

What do you think? Safe to continue or should I restart to avoid any nasty game-breaking surprises?

Roam around in the 1.21 patched world, fight some monsters, talk to npc, play some gwent, etc. and base your decision on those experiences.
If you haven't played NG+ yet, you might want to restart over by starting an NG+ game. It raises the difficulty level and gives you access to upgraded version of witcher armor and weapons, potions and oils.

Patch 1.21 is for the game itself... be sure to also install the updated DLC package and the updated expansions (if you have those.

I can't see why you would have any issues as long as the game, DLCs and expansions are up to date.