Huge groups of deer keep getting aggro's on me, but they don't attack or chase me so I end up having to find and kill every single one on my own so i could fast travel or wait. Anyone else having this happen and do they know why it happens?

Spriggans cast "Call Creatures" on them.

(I've been attacked by goats under this spell)

Maybe it's mating season or something. Shout Kyne's Peace and call it a day.

So have I, but there aren't any Spriggans around.

Change your clothes, something might have sprinkled on them.

But seriously, do you use mods, they might conflict with each other to effects like that.

Or do you use Illusion magic? As the master level pacify spell has a huge radius and should pacify all the dears in a huge radius, probably big enough for you to be able to fast travel.

Console player, so no mods. And thanks for the illusion tip, though.