Chameleon really was imba in Oblivion. I remember once enchanting an entire set of attire early on in the game and thus becoming invincible at level 5 or so. While this was definitely fun it felt weird since some of the enemies I faced would literally just have had to swing their weapon around and hit me once to swat me like an annoying fly (which I, all things considered, probably was). Adding hearing and maybe even smell (although this isn’t all that accurate) to the enemies’ capabilities of detecting you sounds like a good idea.
The spells available in OB required Journeyman (skill level 50) or higher in Illusion before you can cast the spell and thus enchant with it. If you were level 5 and could do this you must have spent quite a bit of time working on your illusion to do so. IF you spend that much time honing your illusion skills then you should be able to do so.
I don't think Chameleon should be capped. I do believe it should be a much costlier spell to use and much more difficult to reach 100%. In OB you needed I think 5 grand souls to enchant 5 pieces to get to 100% and Sigil Stones made it worse needing only 4 of them to get over 100. Since there will not be sigil stones in Skyrim this is a help. But I think at most a grand soul should only give maybe 10%.
Alternatively, since it is just a single player game, maybe chameleon could be gained via very high level perks and still not reach 100% easily but still possible. If someone wants to run around in 100% chameleon suit they should be able to do so. They just don't have to make it so easy as it was in OB. And with one less enchant slot (cuirass/greave combined) it will be much more difficult.
Edit: For Invisibility I think it was fine in OB, but too cheap to cast. As others have said it could possibly be improved by constantly draining magicka while invisible. And it could be kept to higher skill levels.
I don't think either spell effect is bad. It is the ease at which you can get it and bad AI that makes it overpowered in my opinion.