While it's admittedly a bit weird that a Deadra would voluntarily wander in Oblivion, given what the lesser Daedra say about it (though it could be different for Daedric Princes, I guess), I assumed this was because he doesn't have many other places to go. Without a Realm, what else is he supposed to do?
I think that Jyg realized that it was pointless. He knew that he'd barely have the time to destroy everything that he'd become Sheogorath once more before being able to rebuild the Realm as Jyggalag wants it to be.
Both Sheogorath and Jyggalag want to break this cycle. Sheogorath to protect the Realm as it is, Jyggalag to be freed from the curse. Sheogorath's gambit helped Jyggalag understand that to break the cycle, he had to no longer behave as he did every time. So what was the thing he did every time and thus should refrain from doing this time? Retaking over the Shivering Isles.
So, he renounced them. He abandoned them. He's giving you all his madness and all the part of him that is Sheogorath, and will stay away, far away, from the Isles in the future.
But he still kept much of his power. And he's a Daedric Prince. What is he going to do in the void of Oblivion? Why, create a new Realm of Order, of course. One that will be untainted by the history of madness of the Isles.
Now this is but one way of looking at it. Another way is that the cycle is not broken and that Sheogorath was lying, knowingly or unknowingly, when he was saying that such or such thing never happened before. I refer you to Arden-Sul, Agnon, and the myths of the Heretics and Zealots about how Sheogorath is a mortal usurper... Since the Realm is the Prince, what if Jyggalag going in the void simply means that he dissolves and reform as the Shivering Isles? And as such, slowly reemerge as Sheogorath as he melds with and takes over Sheogorath's previous mortal champion?
It's simple, really, if you don't think about it.