Am I missing something? Swinging the camera around is like moving my mouse through mud.

I had trouble too.There is a setting when you hit "esc" I think it's either under video or gameplay control.

Yes. Under the escape menu go to settings> gameplay> look sensitivity. For some reason the default sensitivity setting is way off, at least with 3 different Logitech mouses I have used. Increasing the sensitivity will make your game much more pleasant.

Ah, I didn't think to look under gameplay! My altmer is now running around the world at a more normal pace. Thanks.