So I've found plenty of lists on all the unique weapons you can come across in Skyrim, however I cannot find anyone sharing any unique items out there, so I'd thought I start. Feel free to share what you have found and I'd prefer if you left out the location for spoiler sake. Here's what I've some across:

1. Butterfly in a Jar. (Exactly what it says, use it as a house decoration. I've only found one of these in 142 hours of gameplay).

That's really it, to be honest. Besides some witches head that ended up in my bag (spoiler free), and some red glowing dwemer core that looks awesome on my shelves! I'm just curious to see if there are any other cool random items out there.

Jagged Crown (Temporary)
Skeleton Key (Temporary)
The Black Star/ Azura's Star
Oghma Infinium
Torchbug in a Jar
Gray Fox Bust
Honningbrew Container thingy
Various other bugs in jars
Dragon Priest masks
Crimson Nirnroots (I know, there are many of them)
And lastly: PARTY BOOTS! (Oh, and the clothes)

My fav. would have to be Notched pick-axe not that it has to do with minecraft or anything =p but it "Improves Smithing Capabilities" and Has shock Damage on it, Now tell me, what are you mining with? lol

I have a crown of flowers, courtesy of a certain bride. . .