Nalion's and Arynel's posts on this topic are good. I thought I'd add that in "Morrowind" you meet three Aedric avatars: Wulf (Tiber Septim), John Hawker (Zenithar) and an avatar of Mara. This does seem the exception, however, only granted to the Nerevarine. In the case of rescuing John Hawker and Mara's avatar, it seems like a test -- I think the avatars could have escaped their captors at any time, but they waited for you as a test.
You gotta remember also that nobody else saw them AT ALL, like you said, possibly as an exception to the Nerevarine because he's so special. ask around Ghostgate, and people are like "Old Imperial veteran? You're a nut."
Also I want to say that mantling is hard. I mean, yeah, sometimes it gets kind of obvious if the guy turns into a huge dragon (although I remember right after Oblivion came out there a lot of people figured that was the purpose of the Ayleidic artifact), but let's take the Nerevarine for example. Back in the day, Akatosh killed Lorkhan and ripped out his heart (according to a good amount of folk, anyway). The Nerevarine takes a big hammer and smashes it, kind of finishing the job Akatosh started. Does that count for anything?