I just got back to Sanctuary, I went to the rack and added it. Backed out and I'm still wearing it. I went back in, took it all back out, put it back in and still have it on! When to "my" house while wearing it and dumped EVERYTHING into a dresser and I STILL HAVE IT ON! Only the helmet is gone. Any suggestions?

All you did was remove the armor pieces from the suit. Possibly also removed the Fusion Core, which results in severely limited performance with the suit.

The problem is that the Power Armor frame isn't something you equip and unequip; it's a vehicle you climb into. Dumping the armor pieces doesn't work because you haven't climbed out of the vehicle. In order to do that, you need to hold the E key on the PC. This is the Activate key, so if you're on the XB1, it should be the A button, and the PS4 should be the X button. Hold that down, and you'll exit the vehicle.

NICE!!! Perfect held down A button and she jumped out NAKED! LOL, THANKS!!!

OK, but now I don't get how I left the armor, but I am able to pick up the pieces and the armor is still standing there.

The thing you were walking around in is a Power Armor frame. It's the core of the suit, and has all the basic systems used by all suits of Power Armor. When you equip a T45 Chest, you're not actually equipping it; you're instead attaching it to the chest section of the Power Armor frame. The frame itself never enters your inventory; it's the vehicle you're piloting. The armor pieces you equip while in it simply determine the vehicle's capabilities.

Thanks
I was too embarrassed to ask that question ha-ha

Honestly, as far as the removal went, I actually had to pull up the in-game help. Unfortunately, this was after trying to find the answer in the official guide, so it took me longer than it should have to figure out how to get out of Power Armor.