I've been playing a pure mage Khajiit on Adept level for the last three days (marathon!) and at level 23, with Destruction at 70, I am mopping up the floor with enemies.
I don't use Enchanting. Wait. Stop. Let me repeat that: I do not use Enchanting. Ever. Not only do I *not* use Enchanting, but I don't use Smithing or Alchemy either.
I *do* use the Arch Mages robes, but I had no problem with the game *before* getting those. Prior to those, I used the Necromancer's Robes.
I have Lydia as a companion and I kitted her out in heavy armor and magic weapons. Having a warrior companion is extremely helpful, just don't accidentally blast her when she's on a knee.
I focused on Lighting and Fire and I used perks to increase the damage for those types of spells (I do +50% damage with each type, and dual-attacks stagger opponents). I use the single-word Slow Time shout, runes, and dual-wielded Fire Bolts or Lighting Bolts (those are the ones that stagger). I do not use any physical weapons.
I am a friggin force of nature.
You will eventually regret not working enchanting before.. You will find that after the arch-mages robe, there aren't really any better choices for you. Knowing that, @ level 15, you pretty much have the best robe you will see in game for a long time is pretty bad. It's an okay, robe, but is crap.
Also, you will see a ton of casting gear that is light or heavy armor, but none that is cloth.
A 'pure mage' HAS to enchant to make up for really bad cloth caster itemization in this game.