I have never seen hostiles take out their weapon in Oblivion or Morrowind, if they weren't first alerted to my presense.
Well that's certainly not true, at least for Oblivion. All hostiles in the game had an aggro radius where they'd pull out their weapons and get ready for a fight. That didn't mean though they knew exactly where I was, just that they could sense my presence nearby. The same thing happens with their dialogue, with them saying things "Where are you?" or "Damn rats, always scurrying around in the dark." or "I got to lay off of the ale, I'm starting to see things." If you heard anyone saying those things, then you knew right away they were hostile, and that they knew you were nearby but didn't know exactly where. And all these things would happen well before you were outside archery range.
Hostiles also have the same armor/clothing that non-hostiles can have.
In the default game, I almost always knew exactly that if I saw someone up ahead wearing light armour that they were a Bandit. I even had to create my own mod to add the benefit of the doubt to my game.
The only thing that differs between a hostile and non-hostile is that they have a low enough disposition toward you, and have a high enough aggression value to make them initiate combat (neither of which are made apparent to you, the player).. that's it.
Again, that's not true. Their behaviour is different, as is their dialogue and usually the way they are dressed. If you've never noticed those things, then obviously you weren't paying attention.
That's kinda how you play a stealthy character. What you lack in armor, you make up for in sneak+poison damage. If the game's not going to let a stealth character use stealth to their advantage, it's discouraging that play type.
That doesn't mean that each and every battle ends up in you getting in a sneak attack. The opportunities where you know for certain whether the character is hostile or not are frequent enough that you don't always have to open fire on first sight. It makes the game much more interesting if you have to examine your opponent first and determine whether they're an enemy or not. And sometimes you may get it wrong. That's just a normal part of combat.