Well unless there's something defective hiding behind or around it, I can't come up with any other reason. At least the palm trees aren't making it a further degradation on things now. Perhaps an NPC or creature in the area is causing undo stress and they happen to be in line with the arena?
For me I got a major slowdown when I looked through the central town area outside the Orcrest palace. I was confused by this because there didn't seem to be much rendering on screen...on my system polygon count makes almost zero difference, but the amount of NPCs and light sources totally cripples my machine, since that's mostly CPU driven (I have a E6850 3.15Ghz).