I have recently started a bounty at markarth and no matter what I tell them the conversation just keeps going in circles. the only way to walk away is "I would rather die than go to prison" not sure what to do or is it just something I am just gonna have to deal with for the rest of the game.

Try to commit crimes. Steal something, assault a guard. Then see what happens.

This glitch happens sometimes after the forsworn quest in Cidna Mine.

-The simplest way of resolving this is to go back to the Temple of Talos and speak with the guards inside. Choose the option to go to jail, then walk outside and speak to another guard to pay your bounty, persuade, etc. successfully.

To make this as easy as possible I would use an invisibility potion to sneak into Markarth.

Hopefully this works I know from experience how annoying this glitch can be.

Are you perchance doing the Forsworn Conspiracy quest?

The cidna mine guard bug was never patched?!