I'm not terribly worried, but i'll be crushed if they pull what happened with oblivion. For those of you who forgot OB was supposed to be released 11/22/05 and on 10/31/05 they announce a delay of a few months, so that was 22 days before the release that they pushed it back.
I would be so pissed if that would happen with Skyrim. We're 39 days away from 11-11-11, when it's 21st October (21 days till release.. 21 and 21 oh the coincidences...) I will be more relaxed.