I take it you cannot kill him by means of poisoned apples or some other self inflicted items(maybe weightless hood of damage health)? I mean i used to just kill him with the blade of woe or some other means but i find it a bit uncreative. And I don't think he eats anything down there to have those apples worth giving to him.

I've always had the idea to cast a really high level command spell and make him follow me to the Imperial city's prison, then kill him in front of Caludius Arcadia.