I believe level scaling is a good thing, if it is used wisely and within reason, for example, instead of this bandit's level is = to the pc -1, lets say bandits have a minimum level of 2 and a maximum of 13, because lets face it, they're bandits, they aren't going to be all powerful, but they also are going to at least know the basics, vampires should actually be vampires, use a similar system to the bandits then add vampire abilities, all of them. I also think that there are some places you just shouldn't go at lower level because you WILL die. Storming a daedric ruin for example, WILL get you KILLED. Items with level scaling I think was a bad idea, it makes people hold off on doing quests and exploring dungeons because if they wait until they are a higher level they will get a better item, with that being said if you look at Morrowind the items were sometimes just god-like, and I think they should most definitely be toned down. Also, damage (among other things) should be more skill based and less item based, similar to fallout 3's approach.
So basically yes level scaling is a good thing, however it has to be much more complicated than simply, everything levels with you, because that's not how a world functions. Rats are always going to be around and they are always going to be rats, we have to accept that, at the same time daedra will always be around and be ready to kill anything that stands between them and their goals. Also, not all guards should be able to take on anything and everything, always found that odd that some of the strongest opponents were guards, more so since they always come in multiples.
While I agree with you that leveled items is a bit silly, allowing low-level characters to obtain really powerful items can be game-breaking. It's basically a no-win situation for the developers; if they deny the powerful stuff until one reaches a certain level people will complain about leveled loot, but if they allow the powerful stuff to be obtained early much of the game can be trivialized. There's really no good answer here, unfortunately.
Your creature ideas are similar to the system I would use, which utilizes complex leveled lists as well as minimum levels in some locations. Under my system, scaling is limited to a preset range for a given creature class (Rat, Goblin, Daedra, etc.), and within each class there are 1-5 subclasses which each have their
own level range. Now, these classes and level ranges are
not mutually exclusive; instead of just Warlords and Shamans in high-level Goblin encounters, for example, there could be weak ones as well. There are also a
small number of creatures/NPCs who have a high minimum level, scale indefinitely with the player but at a lesser rate, and are
only found in those locations marked as having a high minimum spawn level. These are intended to keep things interesting for top-end characters with godly gear who would trample all over anything less, and would have markedly better combat AI to boot.
Under my system there would be no sudden replacement of creatures; there could be Rats, Wolves, Minotaurs, and Dreughs all in the same area at once at higher levels, instead of only Minotaurs and/or Dreughs, for example. While I have no provision for doing so it wouldn't be hard to tweak a couple of lists to allow for the possibility of getting something you really shouldn't be fighting yet, although I would suggest putting a limit on just how far above the norm such a spawn could go. After all, it wouldn't be much fun if you didn't at least get the
chance to escape.
NPCs are somewhat more complicated, since there is the added issue of equipment; it should make sense for the faction, rank of individuals therein, and their given specialty. Whether or not it should improve as the character does is a tricky issue: if it does, you end up with every third bandit or marauder in full Daedric, and if it doesn't you can end up fighting through a horde of same and coming out with 6 sets of Fur and/or Iron after an hour of brutal fighting- the latter isn't likely to go over well when the character in question is level 45. Which is somewhat ironic, since a motley collection of Fur and/or Iron armor would be quite fitting for a Bandit hideout, with perhaps the odd high-end set on a leader-type and maybe some good items in the loot chest.