Technically the invisible walls are easy to remove, I've seen them in the GECK around many of the exterior mountain cells.
I believe they exist to prevent the player from entering certain areas except via the path they choose - which I think stems back to Obsidian's days as developing mostly maze-line games where Bethesda makes open-world games. I have found this to be the case in the game in many places, where in Fallout3 the areas were always designed so that the terrain defeated you if they didn't want you going into a certain area, where as in New Vegas it is clear I can make it over certain hills but am artificially blocked. Its not a complaint, just a difference in level design style that I found notable between the two games.
To me this makes sense in the respect that Obsidian prides itself on the strength of its writing and questing, where Bethesda is famous for its world building, engines, etc. It is thus no surprise that Obsidian would care more about forcing the player down certain paths so that the scene fits the story better, where Bethesda would be more interested in making a homogenous world that fit together perfectly and makes sense on the ground. I find the styles are complimentary to how their worlds are built, and if the artificial walls come down that will allow the player to break the story line in places that the devs didn't want us to.
Its a difference you can either get used to or wait for a modder to delete the invisible collision barriers in the game and bring down those walls - but as mentioned several times, there will be unpredictable and likely quest-breaking if not story-breaking and certainly immersion-breaking consequences will result.
Miax