Kinda self explanatory , so do you consider it as a sequel or a spin off?

Sequel.

It has more continuity (both in actual content and in style) with the original series than Fallout 3, on top of (in my opinion) being a better game.

In fact, I'm shocked that it wasn't just called "Fallout 4".

It's part of the main series; too much content to be a side story.

Here's a good example. Leilana's Song DLC for Dragon Age: Origins is a side story. Dragon Age: Awakening is part of the main series. THis is because Leilana's Song is pretty much a small chapter that has limited relevance to the series as a whole; it's about Leilana and her betrayal by Marjoline. Awakening, although smaller in scale than Dragon Age: Origins and Dragon Age 2, deals with many after-effects of Dagon Age: Origins, and has more impact to the series as a whole.

New Vegas, like Awakening, has a major impact on the universe of Fallout: the NCR expands further on a journey to City-State, Caesar's Legion becomes a proper post-war superpower, or the Mojove becomes a successful independent state, able to beat back two major forces nearby. Although officially a spinoff, it's worthy enough to be part of the main series.

Why wouldn't it be part of the main series?

I consider it being a part of the main series yes. Not to mention my favorite Fallout game.

I have to agree with this incredulity. With the size of it, the continuation in theme from the originals, the new representation of factions and the importance of the story there is no reason why New Vegas wouldn't be considered part of the main series.