I've had a decent variety of characters in Skyrim, but this is my first time trying to focus almost entirely on magic. I have one set of Elven armor that I usually carry in my inventory because sometimes I end up in a battle that I just can't get through without the extra protection. The thing is, it doesn't seem very lore-friendly, and even though it's Light armor, it still takes up more of my inventory than I'd like, and my character is a treasure hunter. I was wondering if anyone had any tips that might help me toughen up. I have to get Ebonyflesh, but you don't do that quest until your Alteration is 90. The Bound Bow and Bound Sword have pretty much replaced my need to carry weapons, and I do hold on to a decent stash of potions.

I would suggest conjuring atronarchs and using Illusion spells to whittle down your opponents so you don't have to go toe-to-toe with them.

Enchant gear to give you more health.

Also, bring followers with you.

And don't forget shouts. the Fus shout is good for staggering. There's also a shout that will freeze an opponent solid.

If you're using the one handed sword when fighting, free up your off-hand so you can block. While holding block and clicking the attack button, you can bash your opponent, stagger them, and interrupt their power attacks.

Paralysis poisons are a great last resort. Keep a dagger with paralysis poison on it, hot key it, and use it only as a last resort. I think Paralysis is an Alteration Spell, so eventually you'll be able to forego the poison and just use the spell.

edit - one last thing. If you're running everywhere, then you'll likely end up in the middle the fight when attacked by bandits, etc. Consider walking through new areas so that you don't get surprised.

Don't use conjured weapons if you are trying to play a pure mage, they make it feel like a battlemage. Learn some good destruction/conjurations spells, get a tank for a follower and don't rush in to battle.