I'd like to play a mage with the Atronach birthsign. I know it's not too difficult, but I'm not really fond of using alchemy. So I'm wondering: Are there any mods that add other ways to recover magicka? Other ways meaning, of course, besides willpower or potions?

I don't know of anything that directly meets your needs but there is, as I'm sure you are already aware, ayleid wayshrines scattered across Cyrodiil. http://www.tesnexus.com/downloads/file.php?id=26389 adds these to your map as you discover them making it much easier to find a 'gas-station' when you really need one.

Welkynd stones are a plentiful and non-alchemy alternative to potions. http://www.tesnexus.com/downloads/file.php?id=29007 makes them even more so.