I never had that problem in Oblivion, because we could spend money on training, buying horses, or buying houses and upgrading them in all of the major settlements.
I think this is a serious topic. Getting very rich in the beginning of the game shouldn't be easy. That makes dungeon-diving and the whole thief archetype pointless.
They should reasonably adjust the amount of money we can find in the world or obtain via quests, OR they should invent new ways rich characters can spend their money. I'm sure we'll be able to buy houses and horses, but we should also be able to customize our houses, by paying gold. And there should be expensive clothing and jewelry that are actually expensive, not 15 gold, more like 200 gold or 1000 gold, so our rich characters can dress up like they are actually rich. Maybe there could be charities, or orphanages where we can make donations, or street pickpockets who could rob some of our gold and run away. Maybe we could help improve some of the local buildings (upgrade them) like in AC2 Monteriggioni, helping the town economy or the residents in some way. Anything to prevent our character from getting too rich too soon.