Not in the Elder Scrolls universe. In TES potions (and Alchemy in general) are magical.
Technically speaking, that's what we're told, but if you believe everything you read in the lore, no matter what you do, magic is as much part of existence in Nirn as gravity is to us, really, unless something involves physically casting a spell or using an enchantment, I don't think of it as magic.
Regardless, though, I don't really see how a character could survive without potions in the Elder Scrolls, if they don't use enchantments or spells instead. It shouldn't be too hard to survive without spells or enchantments, those can be helpful under certain situations, but there isn't really any situation where you can do anything with them that you can't do with other things, but if you don't want to use spells or enchantments, the ONLY way to heal normally available, at least in Morrowind and Oblivion, is either potions, or resting, and you can't do that in combat, this might actually be more of a problem in Morrowind as where as just waiting an hour fully heals you in Oblivion, in Morrowind, you need to spend time to rest, many hours if you've lost a lot of health, thus you could end up wasting a very large amount of in-game time if you're adamant about not using anything related to magic, plus, levitation is NECESSARY to getting to certain places in Morrowind, including places that are part of the main quest, and if you don't use spells, the only way to get to those places is through enchantments or potions.
So yes, you can get by without using magic in the game, though I would not recommend trying to play without even using potions.