Go for the PS2, it has more games. And it is cheaper. And did I mention it has a bazillion awesome games? Unless you already have a PS3 (because they are re-releasing some PS2 games for it) or an Xbox (for those cross-platform games/non-PS2-exclusives).
I have a PSP-2000, I've found a handful of games I like on it. If I didn't already have a PS2 and several PS1 games, I'd probably buy the PS1 games for the PSP off the PSN (did that make any sense?). Also, PSP is cheaper now with the price drop. But, most people on the forums seems to feel that they haven't been able to find many games they like for the PSP and it is just a paper weight. Unless you really love JRPGs, you're going to have a tough time finding games you like. I manage to find some good non-JRPGs, but they can be hit-or-miss and there are far less numerous.
EDIT: If you have the money to through around, sure, go get it. But if the PSPs track record is any indication, in a year or so they'll come out with a better model with better hardware. The line up of games looks more promising, but it's still all a big question mark. I mean, two touch screens, one of the front the other on the back? Who needs two touch screens?
I'd be happy if they just added an extra anolog stick to the PSP and gave it the PSP-Go's improved screen. A screen in a PS2 controller, that's all I need for a portable gaming device. And actual physical media, none of that download-only crap like the PSP-Go.