Or even just the town map? I find it tedious to go from, for instance, the Thieves Guild in Old Town all the way to the city limits of The Hub just to get to the World map so I can go to another town.

There should be a map button located somewhere around the skilldex button and character button on the bottom-right side of your HUD. I don't believe you can use it to move between city districts, though.

I see the MAP button above CHA and PIP. But it only brings up the map of the location I'm currently in & doesn't allow me to view the World Map or change locations with it.
Unless I'm missing something, it seems I have to go to the city limits to bypass other areas & go directly to the one I want.

in short, yes, you do. no quick travel within towns.