The reason given for the removal of Levitation after Morrowind was for level design; giving the player that sort of freedom limited the designers in what they could do, since a lot of neat ideas were circumvented by a simple "oh, I just levitated past that". Most people just think of the dungeon "shortcuts" that take you from the end of the dungeon back to the beginning, but it's true; think of how easy Oblivion Gates would be if you could just fly straight to the Sigil Stone. That's just one example, and reason enough why I'm always reluctant to install mods that add Levitation to Oblivion or Skyrim. You also can't forget that with NPCs getting more complicated, they had to separate cities into their own cells and wall them off; another thing that doesn't jive well with levitation, although with some creativity this could honestly be less big of a deal.
On the flipside, they actually lost some level design opportunities by getting rid of Levitation. When it was assumed the player had access to levitation in some form, you could add stuff like the Ministry of Truth floating over Vivec, or Telvanni towers (wizards don't use stairs!). There's also the dungeons of Daggerfall (the final dungeon being of note), but I'd say giving the player as much freedom of movement as possible in procedurally generated dungeons is important, since you really don't have much control over level design. Arena had spells to go through walls and floors for crying out loud, and most people are a bit more understanding when it comes to the removal of those effects.
So, with all that in mind, do you think it was worth it for them to remove Levitation? Do you think the Elder Scrolls would have been better off sticking with level design with that kind of freedom in mind? I personally don't miss Levitation in Oblivion or Skyrim, but I do really like it in the earlier games.