I understand they are quest items. But gtfo out of my inventory.

If I have a house in Whiterun or wherever, than I want to be able to put quest items in my safe chest. They clog up my inventory and as a stealth player, I like to keep my inventory cleared.

He tends to only hold 15lbs of inventory goodies. But certain quest items are kind of heavy and I don't want them in my inventory.

I probably understand why this was implemented in Skyrim. Because in other games people sold quest items and then complained that they got sold or they lost them, but I honestly don't give much.

I want to be able to remove permanent quest items into safe containers to keep my inventory light.

Especially certain quest items *cough No Stone Unturned*

Quest items don't actually weigh anything. See http://elderscrolls.wikia.com/wiki/Quest_Items_%28Skyrim%29, and also the http://www.uesp.net/wiki/Skyrim_talk:Carry_Weight.

Still don't want them in my inventory. Even if the game mechanic says they don't weight anything with the weight. They weight something in my RP.

This bothered me in Oblivion as well. I hated lugging around an amulet for an entire game.