Travel light. Weight of your inventory has a major effect on your speed.
Beyond that, occasional runs (run a bit, then walk until your fatigue recovers, then run some more) will increase your Athletic skill, and allows you to raise your Speed attribute on level-up. I don't relay go out of my way to do that, but an occasional dash when the situation calls for it is sufficient to boost it over the course of the game. By the time your Speed hits 50+, your character should seem to move at a decent rate. I notice that when I create a new character, they always seem to move at a crawl compared to my previous one.
There's an artifact (or rather a pair of artifacts) that one can acquire, via a simple quest, to get a significant increase in movement rate (with an added surprise), if you ask in the Cheats, Hints, and Spoilers forum.