Strange Amulet can get stuck too.
If you sell it to Calixto, it gets removed and replaced with the Necromancer's Amulet, which can be stored at will. This one is a failure of quest design; nothing that happens during the quest indicates that not selling it was a bad idea, until it's too late and you're stuck with the damn thing.
Glenmoril Witches Head can also get stuck.
These are used when curing yourself of Lycanthropy, and should get removed when doing so. As you can only get 'infected' once, picking up additional ones is probably what causes them to get stuck (note: I haven't tested this, so it could just be a bad flag).
Some other items:
The Attunement Spheres are used to open some of the entrances to Blackreach, however I've always had them stay stuck in my inventory despite repeatedly using them to open said entrances. The Lexicon is removed after you return to Septimus, but remains in his inventory after he dies; do
NOT pick either it or the Extractor back up, or they will get stuck in your inventory as their flag is never removed.
The Dragon Claws involved in the Galdur Amulet quest
should change to 'non-quest' status after completion of said quest; I've always been able to store them after it was completed.
Non-journal books stuck in your inventory are generally involved with the radiant quests from Urag gro-Shub in the College, and should get removed as they become a target and get turned in. This applies indirectly to the Elder Scroll as well; he never asks for it but will buy it from you anyway, at which point it gets removed. I have had a couple get stuck until a later collection cycle, though, so it appears the script has a conflict somewhere and doesn't always strip the book like it's meant to.
Bard quest instruments you'll just be stuck with; for some reason the scripts don't strip the items after you retrieve them, regardless of when you actually obtain them. NOTE: this is supposedly fixed in the 1.4 patch, but from what folks are saying here it would appear that getting them prematurely can result in them getting stuck anyway.