There is a ice cave with necromancers that has the gem that is easy to miss. I'm on my mob, and I can't remember the name of the place.

Hob's Fall Cave - which is also a quest objective for Pantea's Flute (Bard's College quest).

[In other words, do NOT go in there to clear out the mobs to get the stone unless you have the associated quest - or unless you don't plan to DO the associated quest....]

Im currently collecting them myself and if I encounter anything strange ill let you all know.

Don't forget the stone which can be found at the destroyed DB Sanctuary, easy to miss.

Have you checked:
-ArchMage's room
-Solitude House
-Yngvild?

I missed the Yngvild one everytime, might want to check it out.
Also, i assume you already got the stone from thalmor embassy?

I have 20/24 stones (PS3). I still need the stones at Stony Creek Cave, Yngvild, Rannveig's Fast, and Hob's Fall Cave.
So if I get the stone at Hob's Fall Cave I'll ruin the Flute quest If I havn't started it? Will getting the stones in any of the
other locations I listed also ruin quests I have not started? My last save is just inside Stony Creek Cave, and then I quit.
I hope I havn't ruined any other quests in my search for the stones. I've just been checking them off one by one on the
wiki page I printed out.

Dont think so because the flute doesnt spawn untill you start the quest. Im not 100% sure tho ^^

No, the flute will be there, but its ok.
You dont have to pick it up.

Yeah I actually just picked that stone up and saw the flute but I did like u mentioned: left it there ^^