I've been looking but can't seem to find where their main base of operations is. Is it Dustman's Cairn?

Gallows Rock.

Ahh, I remember now. That's where Krev the Skinner is, right? But do they hang out anywhere else? I'm guessing it'll be much different if I don't go to Gallows Rock with the Companions?

There'll be silver hands present but the named leaders will only show up if you're on the appropriate quest.