When I press `E`, I can only steal from that person. How can I talk to him?

Sounds like you're crouching. When crouched, you can steal/pickpocket. Press control to stand up and then try pressing E again.