Which dungeons you consider most difficult? I vote these places

Caves

Swindlers den

Liars retreat

Lost knife hideout

Dewmer ruins

Ikrgthand

Kagrumez from dragonborn expansion

Mines

Solijunds Sinkhole.

Military forts

Mistwatch

Treva's Watch

Nodric ruins. Now thats difficult. There are many tough Nordic ruins.I would vote two

Forelhost (I find it the toughest ruin among the ruins that have a dragon priest)

Lost valley Redoubt(the toughest ruing among the ruins that have frosworn)

The dragon priests are specific in which attacks they favour, and so characters with the right resistances will find different ones harder. E.g. A Dunmer with fire resistance will have less problems with a priest that flings fireballs, than one who uses frost. A Nord will find the exact opposite. One of the priests is in a dungeon that's flooded, and has an annoying habit of getting out of reach over the water.

But I'd agree to some extent that those guys make for a tough dungeon. They're a fixed level 50, which can be way above you character's level when you meet them. On the other hand, a high-level character might have more problem with an NPC boss - e.g. bandit - who's levelled to match the player.