I'm trying to get Azura's star, and I need glow dust.

Where can I find one of these little basta%$ds??

Just go to an alchemist and buy it. Claudette Perrick at the Gilded Carafe and Ogier at The Main Ingredient usually have some. If you are in the Mages Guild, there is a sample in the Bruma guild hall living quarters.

The game should spawn Will-o-the-Wisps around the shrine if you are a high enough level, too.

Those will-o-the-wisps only spawn if you've started the part of the Main Quest where you're supposed to get a daedric artifact, and I believe they are actually leveled and non-aggressive - easier to kill at low level.