There were things mentioned in IV about Skyrim. I remember hearing about the Redoran thing. Wasn't there one about Nords boycotting Imperial goods?
They mentioned things about a lot of provinces, actually, if we expected everything mentioned in the other provinces in Oblivion to actually be shown in the game, the game would have to cover all of Tamriel. And in any case, most of those issues are probably either over or irrelevant by now. Though I seem to recall a rumor in Oblivion about the Greybeards talking about the end of the world, which sounds like it might well refer to Skyrim's events.
As for me, though, I always accepted that the Elder Scrolls V was likely to be set in Skyrim, due to that name being trademarked by Bethesda. Though it's curious some would think it would be set in Summerset Isle, since I recall that there was a time when I heard a rumor that the Elder Scrolls IV would be set there too. Not sure why that is.
But what I would really love is a TES game set in the whole of Tamriel. If they could make it as unique and handcrafted as Morrowind. Truly huge, possibly a whole disk per province. Play a Khajit, start in Elsewhere, a Redguard in Hammerfell etc.
So basically Arena with modern graphics and the level of detail we expect from current Elder Scrolls games?
Yes, I don't see that happening any time soon.