Albino Radscorpions. They are moblile, take little damage, and have no weak point.
They actually take standard damage, it just seems like they don't because they have 1500HP and no real weak spots (they do take a little extra when shot in the tail).
My most annoying enemy used to be Bloatflies, since they're not worth bothering with and show up pretty much everywhere. I have since modded the Animal Friend perk to include them, which eliminated the problem. I wanted to use the Entomologist perk for that, but AF is set up significantly differently from the other perks and it would have been a pain reworking Entomologist to suit.
These days, my most annoying enemy would probably be Feral Ghoul Reavers, since they are by far the most common of the 'bullet sink' enemies and on top of that often get a surprise attack in before you can react to their presence. It doesn't help that there are at least three locations necessary to pass through for the MQ that are infested with Ghouls, which at high levels are almost all either Reavers or Glowing ones.