There are many reasons why Oblivion killed the exact immersion you're talking about for me. I'll try to explain the best I can so that forum browsers of all ages, IQs, and platforms will try to at least understand where I'm coming from.
Lets start off by targeting the very beginning of Oblivion. Sure it was all very fantastic, feeling like you were a part of this crazy plot involving the emperor, being let free by the guard from what could have been your life long prison cell. I truly did enjoy the story immersion presented in Oblivion's starting tutorial for the first few moments, but by the time I exited the sewers I was already annoyed with the way it attempted to force you in to playing every style from the get go. There are no barriers when it comes to Oblivion's character progression. If your character has a lock pick, suddenly you can pick every lock in the game if you have a trivial amount of skill at a mini-game. Somehow (in this example) My Orc fighter knows minor restoration spells, and even knows how to shoot a low tier fireball. This is a great way to casually introduce players the ability to start skilling their magic whenever they want, however as you will recall in Morrowind, your character had to learn spells from a Mage if you hadn't chosen magical skills from the get go. This presented a very real and immersing reason to pay a visit to a Mages guild should you decide that you WANT to learn it.
Further to the point, as you'll recall if you'd ever played Morrowind, you started out in a town.
![Surprised :o](http://gamesas.com/images/smilies/icon_e_surprised.gif)
hmy: What a concept, a starting town? It was your first base of operation. There was plenty to be discovered there. Remember back to the axe hidden in the log in the tree? Or the extra stash of gold here and there... an actual interest in saving your money to buy the basic adventuring gear from the local merchant, of which a bulk of the gold was obtainable through the towns various nooks, cranny's, quests and what have you. It was a great starting experience. There was a lot to be done if you wanted to, it was of course all optional, but it really gave that "Okay what do I do on my first night? What do I need to survive" feel about it. None of this came at the luxury of a quest compass, which neatly shows you where most all of your quest items, Points of interest, people of interests or quest hooks are going to be. You actually had to PLAY the game. I know how much you all hate playing long tedious and rewarding games <_<
In Oblivion's beginning you leave the tutorial at the gate of a sewer exit, and you're presented with a few nifty points of interest on your compass just to get you right in some dungeon crawling action! That's great! I really do enjoy the instant action feel. right off the bat I was exploring ruins, and better yet, getting loot! Little did I know I'd be ages away from a town for which to sell it all to, and that's when I remembered... I was to open my map, click on a major town, and suddenly I'm there. Great? Wrong. Exploration was the defining feature of the Elder Scrolls series before, and I was soon learning that I didn't really NEED to do that. I remembered how to get from town to town almost explicitly in Morrowind, traveling the roads to do so, and I fondly remember a lot of the scenery along the way. It was all very unique and had this sense of home to it. Without the compass I frequently missed little caves or places here and there that I would later discover as I passed by again, on my way from one town to another. Every time I found a new one, excitement would fill me to the bone, as I would explore the new site. I had little idea weather or not I'd (Bringing me to another huge issue of mine) be killed by something horrible, or dominate otherwise lowly threats, but it was always that subtle doubt that kept me on edge.
At lower levels there is no risk versus reward factor in Oblivion. I can't head off in to an area with my fresh Oblivion character and expect to make off with something rare, expensive or exciting. I'm forced to spend the bulk of my time in game leveling up to see better equipment, or treasure. Beyond that, I could to some pretty scripted questing, with no real adventure behind it. "head from point a to point b, get this, speak to them, do that" style questing, where if you didn't know where "A", "B", this, "Them", or "That" was, it was all dandy, you didn't have to pay attention to the quest at all, nor did you need to do any sort of research. You simply had to follow the blip on your map, or compass, and suddenly you're on brainless autopilot.
I'm not a Morrowind Fan boy. I simply despised most of what Oblivion had to offer. It took everything I enjoyed about Elder Scrolls, and tossed it for ease of access, or simplicity.
TL;DR
The combination of the poorly implemented compass, level scaling, fast travel, And a lack of use from store merchants, or a solid starting town, or experience is what could potentially ruin Elder Scrolls for ME
If we could have the option to turn off the compass, fast travel, and possibly the level scaling, I'd be happy, you'd be happy cause you can simply leave them on. We can all be happy, we just require the support of our devs. They seem to be stuck in one of those "Majority rules" states.