I've played both (started with vanilla, on PC).
Heavily modded Oblivion is many magnitudes better. Simple as that.
But, if you just prefer vanilla, for whatever reason(s), it's still a good game. Just... nowhere near as good as it can be.

Getting used to mods, and mod tools (e.g., Wrye Bash) can take a little while, not to mention some reading - and patience+perseverance helps some...
My games just don't crash. There are no bugs that I'm aware of. And I'm using several hundred mods, some shader edits, and quite a few ini tweaks.

Suffice to say, it's totally possible to use lots of mods, and have no problems whatsoever. :thumbsup:
So many reasons to try them too - a more lore-friendly experience, new entire lands to explore, small and large quests by the dozen, gameplay tweaks (including difficulty/leveling/magic/combat/stealth, etc., etc.), graphics upgrades you have to see to believe, sound and music upgrades and options, totally wacky stuff if you feel like it, sixy/risque options (if that's your cuppa), realism any which way you like it (e.g., food/drink/sleep/fatigue/crime/communications, etc., etc.) ... and so on.
It's pretty amazing, what you can turn the game into. Either "more of the same", "the game as it should be", "Morrowind-like", "bizarre", "anime-style", "a bit o' sims in the mix", or whatever else you might have in mind.