Who is "most people," exactly? I smell Appeal to Anonymous Authority. Report to bridge for disciplinary action or I will fire upon you in majestic ways.
:-)
"Most people" in this context refers to Your Average Player , who doesn't indulge himself/herself in the lore of the Elder Scrolls and makes little note of the very real distinctions between The Elder Scrolls and other (largely Tolkien-inspired) fantasy media that you and I so very much appreciate.
Your Average Player (aka YAP) doesn't buy games like The Elder Scrolls to fly through space with magic and taste the truly unique visions that blossom from the minds of people like our very own Michael Kirkbride. YAP buys The Elder Scrolls games so that he can create some busty dark elves and hack away at dragons and goblins.
Seriously, space=/=science fiction.
Regardless, space is what most people (yes, I'm using that term again) think of when they think of Science Fiction. That said, I stand by the notion that if Bethesda attempted to create an Elder Scrolls game set far into Nirn's future, the move would prove to be a rather unpopular one.