This was a small issue I had with Skyrim on Xbox 360, but Fallout 4 on Xbox One seems to be much worse as it happens everywhere in the game world. It's not so much the frame rate, but more so the camera stuttering when looking around. For example, if you're strafing right, and begin panning the camera down and to the left, you'll get a really noticeable stutter effect, enough to where it's affecting how I control my character's movement. This only seems to happen when walking and looking. If you stand still and look around, it seems fine. This happens indoors and outdoors as well, it's not just specific areas, it's everywhere.
Speaking of stuttering, the sprinting mechanic while in 1st Person also stutters really bad, but remains relatively smooth while in 3rd. These issues really need to be looked into.