Invisible walls is what is the bad thing. Remove those and make mountains actually unclimbable by nature, not with invisible walls.
Making mountains climbable is no small feat. Daggerfell was easy, just have your camera run up against the low rez textured wall. In Skyrim it woudl require something similiar to Assassin's Creed, a system where you actually grab on to the various rock formations. I think people are underestimating how hard it is to make this in a freeroam world with lots of mountains, all of them with different shapes and sizes.
But at the same time they cant have you run up against mountains like in Oblivion. Skyrim's mountains are huge and well... actual mountains. So if you were able to use the ''z'' walking methode (left,right,left right) to get higher then it would feel incredible weak. In Oblivion it was acceptable because those round rocks werent true mountains anyway.
So I'm ok with ''walls'' where it should be impossible to get up anyway. They arent invisible walls, it is the actual mountain that you shouldnt be able to walk up on and therefore can't. And since climbing is too hard to do I wouldnt mind this sort of ''limitation''. In fact if these unclimbable mountains could make the game better. Morrowind had hills with spike* tops on them that effectively made them unclimbable. This seperated various areas of the world and made the gameworld more enjoyable.
*
Morrowind: http://i517.photobucket.com/albums/u340/Maxymiuk/Morrowind%20LP/Part%20II/Update%20I/28-whatcouldhavedonethis.jpg
Oblivion: http://images.wikia.com/oblivion/images/3/3c/CloudTop.jpg
Skyrim: http://dl.maximumpc.com/galleries/Skyrim/Skryim-Trailer-anolysis-28-Mountain-Vista.jpg
As you can see, Morrowind and Oblivion both ''elevated'' the terrain to create hills. Morrowind added spike-rock objects to them to make them impassible. Skyrim will have actual mountains, so it makes sense to make them impassible every now and then.