Un the E3 presentation after he picked up the pip-boy he started "inspecting" it.Is it still in the game?

That happens when you are leaving the vault. Pretty early on and it wont move again.

On PC if I click and hold right mouse and move my mouse up and down it moves the pipboy around. Not sure on controller though.

Edit: To be clearer, it moves my arm around.

If you wait without doing anything, he'll look over it, bang on it, etc after a little while. You can zoom in and out using the back button on the Xbox One controller.

Try holding that button and move one of your sticks.