For me, the question is "what does it add?" I don't see guns as adding much to the game, as their position is held just as well by bows, crossbows, and ranged magic. A gun primitive enough for the ES setting would be more of a hindrance than an advantage, as they have horrible range, take a long time to reload, and aren't accurate in the slightest. You can't introduce gunpowder without introducing the changes it brought to warfare, armor, fortifications, and society in general.
I disagree about the disadvantages. Their range is not much less that an arrow, although the accuracy may be less at that range. Speaking of accuracy, while a master archer will be more accurate than a master musketeer, amateurs are much more easily trained with guns (and crossbows) than traditional bows. And you certainly can introduce guns without the inherent "real world" changes, because, this isn't the real world.
My biggest disagreement with guns is just for the "design" of the game. I don't think it properly fits in with the series. However, if I were to make an RPG, it would be very tempting to include gunpowder weapons.
For one, they are more powerful at short range. It might be hard to take out a charging bear with one arrow, but one .75 caliber musket ball has the stopping power to do so. You get a larger number of possible characters to roleplay, such as Imperial Witchhunters styled on the Inquisition and Conquistadors, pirates, and characters from the famous Alexander Dumas novels.
However, in order to limit their use (so that a majority of the game still takes place in a medieval setting) I would suggest several things. The guns themselves should be expensive. On the level of daedric weapons. (Also, money has to mean something, unlike in Oblivion where it grew on bandit shaped trees.)
Also, they should need repairs often, become dirty or broken after only a few shots. The ammunition (gunpowder at least) would be very expensive as well. A single arrow might cost 1 gold, but a single charge of powder might cost 20 or 50; once again, money must have real value.
Finally, the powder and the gun should be vulnerable to the environment. If you fall in a river or are attacked with water based magick, it may not work. This requires several factors which Bethesda has never touched on, the environment itself and then game mechanics to get around it (such as boats)
So, your character may have an advantage one minute because of his powerful weapon, but upon fleeing from his enemies, he may risk being over taken searching for a boat or bridge to cross the river ahead of him, or risk damaging his advantageous weapon for a more speedy and stealthy get away by jumping in and swimming.