So, I get my stuff out of the treasure chest and go to the door to leave.
Even though I am hidden, I get a bounty of 100 when activating the door.
Then the guards outside the door or outside the building will force me to stop, due to my new bounty.
This makes it utterly impossible.
Someone please tell me there is a way around this.

Can anyone help?

P.S. I hate people that bump.

You can run to another Hold and wait there a week. That way you will eliminate your bounty.

Might want to look for alternate exits, not the front door.

Or run for it.

Waiting a week will get rid of your bounty?

The jails usually have a secret escape that you need to find, once you get out of that there should be a hole in the wall or a grate next your the chest with you stuff. Pick that and get your stuff and you should not get into trouble because there is no way anyone could see.

There is always a hidden exit in the jails. For example in whiterun there is a loos grate(novice lock) leading to a tunnel system. All escape routes are marked by a theives guild shadowmark.

I'm curious about this. Is this true?

Would I have to be in the thieves guild?

Transform into a werewolf after you pick the lock. Any mayhem you cause as a wolf is not added to your bounty (unless they see you turn, maybe).