I'm using an xbox one controller as well. It's mapped pretty well. One little trick I didn't know is that you can actually zoom in on your pipboy so it takes up more of the screen by pressing the select button. For mouse/keyboard users you just need to Right click.
Talking about the right click on pip boy, I've found that click-hold right mouse whilst dragging up and down will move the pip boy around. I'm not what function this serves, just thought I'd mention it.
I've recently got a controller and gave it a little go, I've been K&M for years so I found it rather awkward. I think I might give it another go as I keep hearing good things about the controller. In my brief time with it though I did find settlement management a lot easier to navigate with controller than K&M.
Interesting about the pipboy. It could be to center it on the screen or something. I might mess around with that later.
The only things I found that were easier with a keyboard and mouse was facial customization because dragging is more intuitive with a mouse than with a joystick and exchanging items between you and an NPC/container because you could simply click on their inventory rather than toggling it with the trigger.
I do not have this problem
Like, ever
KB/M is more smooth for fps gameplay but hey to each his own