Check your fire spell, make sure it's up to par with your level. Also, look into shock or cold spells. Most daedra are resistant to fire, and many weak to shock. An enchanted weapon will help you by leaps and bounds, even if it has like, charm, on it, as many daedra have a resistance to nomal weapons, which an enchanted weapon isnt effected by. Do you focus on Magic or Blade as your primary offensive attack? If you level both out evenly, you will be hindered greatly, more and more as time goes on. Been always putting points into endurance? It isnt retroactive. you need as high an endurance as possible in as early as possible to have a strong pool oh health. What enchants are on your armor? This can also make a difference.
Hm. Thanks Ashwanderer...in answer to your question, my blade is far below my Destruction (Blade is at 38, Destruction is in the 40's I think)
I prefer magic as a primary attack, but I like blade equally...that's why I went with spellsword and I love it. Most of my level-up points go into willpower (I think), endurance and something else. (strength I think).
I'm almost about to level up - one more cave run and I'll get to level 8.
Bizarrely, I have "Greaves of the Flame", with about a 5-10% fire resist, I think. It's not much and it doesn't help against the Scamps.

And my boots give me some extra 5 points in athletics. And I have an alteration ring that gives me 5 extra points.
I have a Shield spell - 5% though, so it's not worth much, I can use the Nordic Dragon Shield spell, but that only works once so it doesn't help me much.
I have only about ~500 septims (I spent 4K on a nice horse

)...so I can't afford an enchanted weapon and I haven't stumbled onto one yet; I'm not in a Mage's Guild, but I was going to join the fighter's guild if that matters.