Can anyone confirm whether this quote from http://www.uesp.net/wiki/General:Playing_DOS_Installments_under_DOSBox is true on both 3dfx mode and software mode?:

IMPORTANT: While it is possible to run Redguard under DosBox more or less if you play with the configurations long enough, the game will crash for sure if you enter Clavicus Vile's realm. This crash is caused by the core DosBox is emulating. None of the various emulated core-types can prevent this (verified for DosBox v0.73 and 0.74). At the latest at this point you won't be able to proceed with the game. If you don't have access to a true DOS or Windows 95/98 machine there is nothing you can do about this.

I've installed both versions of Redguard and played them in DOSBox, however I never played it far enough to reach Clavicus Vile so I can't verify it. If it's really an issue, then it should be reported to the DOSBox team.