Well in all fairness Fable's world wasn't exactly like ours either, but anyways I just thought it would be an interesting thought. I mean Tamriel is a civilization, and civilizations change eventually right?
It certainly would advance, unless of course any civilization that went and advanced decided to go Dwemer on us again, but I doubt it. The "Advanced but extinct race" concept starts to lose its credibility when it happens too much. But what would it look like after that advancement? It's hard to tell, logic would tell me that mundane arts and sciences would take similar patterns to Earth, as regardless of what people will tell you, Tamriel might not be Earth, but many aspects of it look a lot like Earth, so I would expect the progression of technology to also emulate that of Earth. But Bethesda will probably want to preserve the "swords and sorcery" theme of the game, so they will either avoid answering this question, or have the techniques of making swords improve, but still have people continue to use swords.
After reading Proweler's answer and MD's, it occurs to me that we might see less advancement in magic at the same time that we don't see technological advancement; it would be like our medieval age, when the Roman advances in sanitation, engineering, and literacy were virtually lost (except for monasteries and the Arabs helping to preserve knowledge and making advances in astronomy, etc).
Those things happened in the real world due to historical events, however, if history doesn't cause similar things in the Elder Scrolls, one would expect magic and mundane technology both to continue moving forward, it might be slower, depending on what happens, but unless something actually causes knowledge to be lost we certainly can't expect to see anything go backwards.
Probably not. Considering how widespread magicka is, they really wouldn't need guns. Also, they have to discover gun powder, which in our world at least was a accident that didn't see weaponization for a significant amount of time. My guess is that enchanted staff weapons will come to prominence, or some other sort of standardized enchanted weapon.
If magicka was widespread enough to make guns useless, people wouldn't need swords or bows or armor or keys (at least not mundane ones, maybe magical keys for magical locks.) or stairs either, but those things still exist in the Elder Scrolls. Though those who can use magic to replace them do forgo many of those things (like Telvanni just levitating instead of using stairs.). I suspect that magicka is not really quite as widespread as the games would make us think, everyone being able to pick up simple spells and become good at magic with simple training is probably just a game mechanic.