you're not seeing the big picture, there's lots of "in between" combat moments, especially in dungeons, so that in between time where the next set of enemies is coming up, that regeneration improvements make a HUGE difference, so it doens't matter if its only "outside" of combat cause overall in a dungeon run, its a real big advantage. and its very noticable, so the fact that its only "outside of combat" isn't important really.
Er ... not so much, if this was truly the case then why not just always switch to armor in combat and then switch back to robes out of combat? +regen DOES work in combat. go ahead and test it yourself. take a character, give yourself a few thousand health and magicka using the console, burn all your mana away, make a save, and time how long it takes to recover with 0 +mana regen gear vs some +mana regen gear. I tested with a simple +75% and saw a recovery rate of ... well big surprise, approximately +75%! ~9 seconds to recover without it, and ~6 seconds with it. And ... amazingly, this works with any amount of mana, proving that mana regen is a % of your mana pool/second, and that +regen gear increases that rate. In combat, it is 1/3 of your out of combat rate. This is all verifiable quite easily.