Early game on survival is really the only time things sponge... Enemies don't have more HP, its just the way survival works (ie: similar to Skyrim difficulties).
You basically only do a fraction of the damage you normally do... Like there is a behind the seens % modifier that literally just makes you only do like 25% of normal damage... So your 12 damage pipe gun is instead only doing 3 damage...
Throw in that enemies in survival have extremely high base damage thresholds... So your 3 drops down to a 2 or even a 1... So that raider still only has 20hp, but you are realistically only hitting like 1 or 2 damage per shot on them. That's why the first Deathclaw sponges so well with the minigun, because it has like 300 hp and each shot is only hitting it for like 1 damage...
But as you get into the perks an get stronger weapons it evens out.
Especially if you go stealthy rifle with silencers route. A fully upgraded/perked out silenced Gauss rifle with two shot has 679 base damage... Full ninja is 3.5x... Full sandman is another 1.5x... Deacon's perk is another 1.2x... Head shots are 1.5x... We are talking upwards of 6000 damage for a sneak attack head shot. Even only doing 1/4th of that on Survival, you one shot Behemoths.