Yeah having one gem I have to go outside and refill every time I need a new enchant isn't happening. Giants don't give grand souls.
I have found an easier way. Most necromancer lairs have an enchanting table. Soul trap a necromancer in the Black Star, make your way to altar, rinse and repeat. A great way to level enchanting and conjuration, as well as making money. Haven't figured out if you can soultrap raised NPCs yet, though.
On topic: I think Mammoths are the best bet for grand soul farming. Dragon priest and higher level falmer are also possibilities.
[link]http://www.uesp.net/wiki/Skyrim:Souls[/link]