Yeah invisibility doesn't mask sounds if I'm remembering right.
Exactly. If you make noise, bump into an enemy, run, etc. they will know where you are.
For a wizard, if you don't use the Sneak skill, you should use the Muffle spell to make yourself silent and the Invisibility spell so they can't see you. Viola! "The Wizard Stealth" combo!