I seem to remember that magic was terribly important in order to "breeze through combat" in oblivion, but my memory's getting swamped after playing so much skyrim.

Would the ol' sword and board be viable in oblivion? I think all i ever played was as a mage..

You bet. Nearly every character I've ever played in Oblivion has not used any magic whatsoever. Most of them have been melee claymore-wielders who did not even use a shield. None of my characters have ever worn a helm either, for that matter.

In Oblivion I have played characters who did not use any spells, scrolls, potions, varla, sigil or welkynd stones, rune stones, doom stones, did not do Alchemy, and did not enchant or use enchanted equipment.

I've played this way in both a vanilla (easy) and a modded (difficult) game. It can be done!

As Pseron said it can very well be done, but for me magic is too fun to pass up.

I find it hard to play without restoration.

There are several alternative approaches. There's the obvious Alchemy, but if you're including that as "magic," there are still ways to get by.

The most obvious, but probably least used, is simply resting between fights.

Praying at shrines. In the Deadlands, using the fountains.

And there are Birthsigns with health restoration effects, too; The Ritual gives you a big healing (200 points) once a day, while the Lord gives you a Lesser Power (it does use Magicka, though, so it's not purely "non-magical.")

Its viable to a certain extent, but you'd probably at least need Alchemy. Personally however, I'd find playing without magic to be boring. Pure combat or pure stealth gets stale after a while.

Not really. I've done Alchemy with two characters. One was my only attempt at a mage and the other was a warrior. I can't say I felt either of those characters were easier to play than the ones who did no Alchemy at all.

I played a dead-is-dead warrior, using absolutely no magic, alchemy, potions, or enchantments through the first half of the Main Quest. The character died in Azura's daedric quest, when she was overpowered by three vampires together.

Playing "Ironman," it's pretty difficult to keep a no-magic character alive. I wouldn't think it would be a problem if you allowed yourself re-loads for those times you get unpredictably mobbed by a group.