I've never flown a plane, or intend to. Maybe since you're a pilot(!) inversion sense to you, but really up going down and down going up doesn't make more sense than left going right and right going left.
I play games without inversion.
Never been a pilot, but always wanted to learn how to fly a plane, that is why I love simulations so much. I believe someone explained it.
Look at it as the joy stick (back when we called it) or controller. When pulling the joystick back, you head moves back so you are looking up. If you are pulling the controller forward, your head moves forward so since you are looking down.
I can't remember exactly how it works, but in a plane when you pull back on the controller, you move a certian part of the plane, (forget the name, the flaps on the wings) so it moves the flaps up or down (again I forget) so hence why the plane moves up, or really points up, but the momentum feels like you are moving up.
To me, it's more of the controller controlling my head, If I turn the controller or mouse left, I point left. If I move it right my head moves to the right. If I pull the mouse or controller back, my head moves back so I am looking up and when pulling mouse or controller down I move my head forward and looking down.
Try this. Pull you head forward, are you looking up or down? I guess you are just self taught that up is up, down is down, while I taught my self first flying a simulation plane, then when in first person games, that the mouse is my head. So if I pull back head is turning back so looking up. I can't do this if I pull forward, I am not looking up but down.
Hope this makes more sense.
One thing I can't do is the left is right and right is left. For me, left have to turn left, right I have to turn right. I should be able to but my brain will not let me do that lol.