The impression that I get (through play experience) is that you can only fit one soul to a gem. So, if you have a bunch of grand gems, and soul trap a bunch of crabs, you'll have a bunch of grand gems with lesser souls in them, rather than maybe a couple full grand gems. As a result, this pretty much ends up wasting your grand gem (AFAIK, you can't dump the souls without using up the gem). This is kind of inconvenient, but whatever. My question is, is there an easy way to tell what kind of soul a creature has before you try to trap it, so that you don't run the risk of wasting a good gem? Thanks.

Generally speaking, the weaker and lower level a creature is, the lower its soul. Mudcrabs, rats, etc. have Petty Souls.

The best plan is to carry some empty Petty and Lesser soul gems, which are readily available. Soul Trap will always put the soul into the smallest soul gem it will fit in.

I think there's a guide on the wiki somewhere that helps with this, if you can remember what the general boundaries were.

http://www.uesp.net/wiki/Oblivion:Souls

The Azura's Star is great to use if you want to figure out what enemy equals a particular soul.