By that time in the MQ the wilderness gates have been triggered, and those are annoying as hell to leave around. Plus, knowing the threat to the world is still present (the amulet's been taken, and that's needed as much as Martin is), not everyone could just walk away and leave it to chance without trying to help.
Some do. The entire remaining defense charges into the fray to help you get into the gate (and then to the Chapel), other people have attempted to go into the gate to shut it down, but haven't come back out yet (so there's possibly help waiting on the other side; and there actually is if you're quick enough).
Everything you've been told is it's not just the Septims they're after. It's an invasion by Dagon, not a mere assassination. As for "Just" got out of prison, I'll blame the world scale on that. The trip from the Imperial City to Weynon Priory, then from the Priory to Kvatch, would likely take weeks in-world at least.
I'm sure adventurers do what they're told all the time. They don't go in that cave when they're told to stay away from it, they don't dig for information in situations they're told to keep their nose out of, they don't go into areas they're told are too dangerous to travel alone.
Besides, like I said you of all people know the consequences of just going away. You were told what's going to happen if you leave it alone, and you were told how to stop it. I can't see any right-headed character listen to being told to go away unless they want the world to end.
Because game mechanics. Quests and plots don't advance without you acting, and TES games haven't really made you suffer consequences from dallying since Daggerfall.
Mechanically, sure, you can stop at any time and it will still be waiting for you to pick up when you want. But if you're role-playing and following the story, it gives pretty much no reason to take a break from the MQ before it's done. Both the Emperor and Jauffre tell you an invasion is coming and Martin is needed to stop it. And Jauffre tells you whoever is responsible likely knows this too and is actively seeking out Martin as well. By the time you get Martin, the amulet is taken, and that needs to be retrieved with just as much (if not more) urgency.