Please help! I'm doing the quest discerning the transmundane. I'm working my way through Alftand ruins. At the bottom of the first area I need to zone into the Afltand animonculory but the door does not show as a zone but rather just a door to open. When I active it, the door seems to open a bit and then stops and makes a clanging sound. It then registers as being open such that I can close it. I can see my quest marker over the door for Discerning the Transmundane but I can't proceed.

This is a critical quest in both the main story and dawnguard and it initiates at the appropriate point in dawnguard if you haven't completed the main quest to obtain the key item. Both require the quest complete to find the key item in their stories. Is it because I'm doing dawnguard before the main quest? I did notice some things in the quest are a little messed up like it says I need to find the key item for Parthurnaax even though I haven't seen the grey beards yet.

Can anyone help? Is there a standard way to debug a door like this?

I assume you don't want to progress the main story for some reason?
I don't own any of the DLC so I can't be of much help maybe post over on dawnguard tech support as well.

Did you try reloading a save from prior to entering the first zone (Glacial Ruins)? You'll have to replay it, but it may fix the glitch.

I completed the quest to retrieve that Elder Scroll by following the Dawnguard questline - I was further along in the main quest than you are, but still before the point where I had to bring it to Parthurnaax, and everything worked just fine. When you get to that point in the main quest it should just auto-update to complete. You'll be stuck with the Scroll in your inventory until then.