Does the Intense Flame perk also increase damage to fire spells or is it just fear only?

Fire spells and I believe Fire damage on weapons.

It just adds the fear effect.

For damage boost, you have +50% Augmented Flames (2 perks).

You can further increase the fire damage with Aspect of Terror perk in Illusion. (+10) This is a very nice perk for boosting fire damage.

More accurately it activates the fear effect. Fire spells are tagged as fear effects which is why aspect of terror adds +10 Damage the fear effect is simply inactive until you take Intense Flames.

Does it over-power fire magic? I wouldn't want it to as I think destruction is slightly OP (im in a small minority, i know)

Aspect of Terror doesn't have too much of an effect on most fire spells, e.g. taking Firebolt from 25 to 35 or 2xAF Incinerate from 90 to 100. The biggest effect it has is on spells like Flames, Wall of Flames and Flame Cloak which do x amount of damage per second. As you can probably guess, the difference between doing 10 point of damage a second and 20 point of damage a second is quite significant.

Its biggest effect is probably on a certain DLC spell that normally does 4 point of fire damage per second for 15 seconds (4x15=60). WIth aspect of terror it does 14x15=210 damage over the 15 seconds, even more if you take Augmented Flames and use another certain object from the particular DLC. But this spell in question requires unique circumstances to use, i.e. it's not just another spell tome spell, so this may or may not bother you.

Short answer - no, it doesn't overpower Fire Destruction, coming from another user who doesn't feel normal destruction is at all underpowered