I've found this bug is very common while I looked for how to fix this.
When I enter Calelmo's lab the wizard's guards are hostile although I got the key peacefully by killing Nimhe for Calcelmo.
Upon seeing me, the Markarth guard adds 5 bounty on me and asks me to pay for it. I pay, get teleported, then get back, and the same exact thing happens.
While in Calcelmo's lab, my follower (Lydia) tells me "You're not supposed to be in here!"
This is frustrating me, please try and help...
Thanks