I switch off without even thinking about it usually. Much of it actually has to do with game mechanics though.
Unless I'm using a scoped weapon, my accuracy is terrific simply because holding the right mouse button down gets a combination of an unobstructed view (usually) of the target with a single dot representing the point of impact. "I can shoot the wings off a gnat at a hundred paces!" but only in third person. the reason I inserted that "usually" is that if I'm in third and some object is close behind me, the view is obstructed by the back of my character's head or the inside of the face...
I use first for gathering up loot and clutter that's laying around because (Perhaps this is true for you as well?) in third person it can be hard to pick up items because the cursor often doesn't bring up activate option for the item, and it also can be hard to see through the character's body and see everything that is in range to grab.
I'm not certain it's immersion, but I often feel in third person I actually have a closer in-story connection with the character and what he/she is doing at a given moment.