You can sell them, and there is a quest in Whiterun where a woman needs one.
Yeah, i hate that too, you never know when you're gonna need something stupid for a quest and only find out later after you sold it.
I'm holding onto a few flawless gems also fro that very reason.(just in case)