The only thing I think you should have to do is remove the full face helmets, as it is not easy to interact with someone when you can not see their face. Especially when you go into a shop or inn. The rest of the armor is good to me, but yes NPCs should have some reaction to it.

There should definitely be a consequence to wearing certain sets of armor. I still believe Fallout: New Vegas is the best and most accurate example of how things could work in Skyrim. Whether BGS adds in the whole faction system is still in question. But I believe it could add a lot to the game, and definitely make people consider what they wear.

Ok, i am a hardened mercenary looking for work. Why would I possibly not wear armor to help advertise myself?

Let's say you need help. There's a cave full fo trolls near your farm, you need those trolls dead or they will eat your kids or something. Now tell me who you would rather hire to kill these monsters:

1. Some random guy walking around in regular clothes like everyone else.

2. A guy walking around in worn and aged armor, stained with blood, scratches and dirt. With a sword on his back and battle scars to prove that he's got experience.

I'd go for number two.

It should depend on the type of armor. Why would they hate you for wearing a full set of steel armor? They'll just think you're an adventurer. Hating you for wearing Dark Brotherhood armor? Yes, I can understand that.