Revert to an earlier save.
Heh, well I was trying to avoid this because I would have lost hours of playtime.
You could try using SetJournalIndex "A2_1_MeetSulMatuul" 0, but that won't fix your journal, and you'll probably have to enter on more than one occasion since it's not a real index for the quest, and from what understand, won't get saved to your game.
This seems to have done the trick. I was able to get Caius to close the Gra-Muzgob quest and move on to the Vivec Informant quest. The fix seems to have saved to my game - after loading the subsequent save, Caius is still on the Vivec quest. I don't care if the offending quest entry isn't removed just as long as Caius is dishing out the quests correctly.
I had no idea you could set a quest to 0 so thanks for the help!