I also have a key to the well for the Dark Brotherhood. I can't get it out of my inventory either.
I also have 12 Rumare Slaughterfish Scales that I can't get out of my inventory.
I also have a key to Mania and a key to Dementia in my inventory that I can't put anywhere.
Yesterday I was dorking around and accidentally chose to go to jail for a 5 dollar fine. What I noticed upon my escape from prison was that when I reclaimed my items from the evidence chest, none of these items were in it and I didn't get any of them back. It's kind of worth it to me then, if that's the case. I didn't save yesterday, but I was tempted to.
I've opened the door to Mania and the door to Dementia. Do I need those keys ever again? What about the key to the Dark Brotherhood well?
Would I be making a huge mistake if I do this to clean up my inventory?