Would there be a DOS emulator that would work on PSP without using custom firmware? (From what I understand that is illegal). If so would Daggerfall even be able to run on the PSP?

You wouldn't be able to play it. Two reasons:

1. Not enough buttons on a PSP.
2. Melee combat uses mouse gestures, which would be next to impossible to emulate without a touchscreen.

DaggerXL on PSP is definitely possible, a DOS emulator is not.
Also, Aliotroph, how hard is it really to press right trigger and drag the anolog stick in a direction?
Edit: Custom Firmware is absolutely not illegal.
I'm guessing you're thinking of piracy, since custom firmware is the only way to go about doing that on a PSP.

It's illegal if it's a modded copy if Sony's firmware.

Yes, dragging the stick and holding a trigger is fine, but you need to be able to do that while also moving around or you're dead. Then there's the lack of anologue sticks...

I suspect DaggerXL could work, but with a vastly redesigned control scheme. The Vita also looks more promising, both in terms of its computing power and because it has anologue sticks.

Sums up my opinion pretty well.