Just out of curiosity, have you played a TES title before Skyrim? There's two parts to the term 'RPG' - the first part is 'role-playing' and the second part is 'game'. Together that makes an RPG a game in which you role play. And that's exactly what Skyrim is. You role play, as if you were actually the Dragonborn travelling through Skyrim, which is the reason why you might be lead to think it is a terrain exploration simulator. That is at least half the fun of role-playing games, exploring the world in which your character lives. This has been the case ever since pen and paper RPGs. I think it is absurd that you would think Skyrim not to be a game. This will be even more absurd when Skyrim wins Game of the Year, because then it would just be 'Skyrim: of the Year Edition'.
I played Daggerfall, Morrowind and around 100 hours of an extremely heavily modded Oblivion...which was still too awful to complete. I've played literally every single console RPG from the nintendo era until now as well as pretty much every "big" CRPG since the first Wizardry. So yeah, I know what makes an RPG, and the TES games have remained completely stagnant in virtually every way other than graphics. Actually, that's not quite accurate. They've actually regressed in most other areas.
1. Moment to moment gameplay is still awful. It's circa 1994
2. There's no real incentive to do anything past the main quest because it's all the same Fedex stuff over and over again.
3. There's much less "roleplaying" than in the previous TES games unless you do most of it in your head. The perk system is way too limited as is the removal of spellmaking and many of the various spell effects. There's barely any options in dialogue at all, and there is almost never an alternate way to complete a quest. You either succeed or you fail, there's no alternative.
4. Most of the game revolves around exposition; you basically stand there listening to what people have to say, or read a book. The rest involves you traversing the interesting landscape doing mundane and repetitive tasks again and again in the same repetitive dungeons, gathering the same repetitive and boring loot.
Your "logic" about why it's a roleplaying game can be used for just about every single video game ever created where you take on a seperate persona. Thus, Super Mario Brothers is an RPG.