TES 1 took place in all of Tamriel. The entire continent. You visit every province.
TES 2 took place in the Illiac Bay. This was most of High Rock and the northern part of Hammerfell.
TES 3 took place in Vvardenfell, a frontier district of Morrowind.
Do you see the trend here? With every instalment, as the game grew in detail (and processing requirements!) the scale became smaller. The devs knew that they couldn't take all of Tamriel and make it on a scale as detailed as Daggerfall. They knew that they couldn't take a province and a half and make it on a scale as detailed as Morrowind. So they adapted.
Here's where the problem comes in:
TES 4 took place in the Imperial province AND one of the planes of Oblivion
This was not good. The game became even more intensive, meaning they could put even less things into the game, and yet they decided to make the scale larger. It does not work like that. What ended up happening is that Cyrodiil became this ghost province that could be crossed in an hour and that had maybe all of a hundred citizens. I understand why - with all the new radiant AI technology they couldn't really be densely populating their world. But that should have sent off a warning bell saying "Hey, maybe we should aim for a smaller region." Morrowind was more populated and Morrowind took place on this scarcely populated island.
TES 5 took place in Skyrim. This isn't quite as bad, but it's still far from ideal.
I maintain that TES 4 and 5 would have been much better games if they had taken place in a region within their specific province, rather than trying to show the entire province.
See, if it were up to me, I would have had Oblivion take place in the Imperial City and the surrounding countryside. Cut out all the other cities, except maybe Bravil, and instead toss in some towns and villages. Make the Imperial City actually feel grand and majestic. The game would be the exact same size (well, I'd prefer it to be bigger, but you know...) but it would be using that size on a scale much more suited to it. The game world would feel more alive and more compelling, and it's the same size so their's still just as much exploration, it's just taking place in a smaller region of the map, making it a lot more believable. Also I'd set the damn thing in the jungle, like it was supposed to be, but I digress.
Skyrim? Same thing. Maybe have it centre around the border, maybe not. But the point is, a chunk of land that's not supposed to represent all of Skyrim, but that instead is going into a smaller region in more detail.