My proposed change is that instead of forcing you to keep your items and making them weightless, they should have a small weight, and instead of not being able to drop them, have a message appear every time you want to drop something important. The message would obviously be something like, "WARNING--This item is important to a quest that is currently active in your journal! Are you sure you want to drop it?". That would make everything better as far as clutter goes. And the warning would be sufficient to keep you from accidentally dropping the Amulet of Kings.
If anyone else has a way to keep the important items from staying in your inventory, I implore you to share.