The (working) Power Armors have servomotors, meant to alleviate the heavy weight of the frame. And Power Armors (aside from Salvaged ones) require training to use them. As in, you have to be proficient in utilizing those servomotors before you can equip one.
This means, while the armor itself is heavy, you won't feel all the weight when you're wearing it: the servomotors are doing all the job of moving the armor itself around.
So, it's the difference of pushing a car and driving a car. Or, at least the difference of pushing a car with your hands and with a bulldozer.
Wouldn't it mean that, from the wearer's point of view, Power Armors are pretty much as restrictive as Medium Armor? Or, maybe even Light Armor?