Yes I have two of them am afraid they are STUCK forever in my inventory because I paid gold to the temple not to clean them all up after I already picked up 2 of them.
So that part of the quest is "done" but the "remove quest item" part did not run to get rid of them.
And here is the really bad part, quest items in this game HAVE WEIGHT!

This could eat at my ability to carry things if this happens a lot. I thought that is why quest items did not have "weight" in Oblivion, because you were not in control of dropping out of your inventory.