The OP may not be able to buy that house though. He's avoiding quests, so it may take awhile to build up enough money (25k) unless he takes advantage of certain glitches.
I'm curious though, how much stuff are you planning on storing? If you upgrade the house in Whiterun fully, there are a decent amount of containers. You could take advantage of a few (like 3-5) containers in the College of Winterhold by simply passing a test (to gain entry) and then take a tour of the place.
Highly possible just raiding bandit mines and tombs... No exploits needed.
Once you take into account... most things you find at low levels is not storage-worthy in the first place.
Quest items, if found, can not be stored, and will linger in your inventory until you get the quest, which may be broken, by just finding the item before getting the quest. (Avoiding quests, yet obtaining a quest item, will break more than getting the actual quest, and doing the broken quest.)
The house in Whiterun does not require doing anything other than talking to the Jarl, then talking to the guy selling the house. I don't recall any associated quests with the purchase.
If you buy the house furnishings... in Whiterun. That gives you three chests, two end-tables, a barrel, a cupboard, and a satchel... You have more storage than most shop-keepers have.
The least critical quests are the MISC quests, which still break, when they fail to clear, or fail to create/spawn the random item you are sent to find. Or you get the item, and they fail to remove it. Many of those will just build and build and build, until you have a bunch of things which never go away.
Just assume the whole game is broken, (though it is only like 5%), and use this character as a learning tool, or a scout. Find your play-style, that you like, and build your next character, when the patches are out, with your collected knowledge.