It's my understanding that there are other routes between Skyrim and Cyrodiil, but that the Pale Pass is pretty much the only major route (and the only one suitable to march an army through, coincidentally). That said, I've never really paid attention to the issue, and my understanding was shaped by a discussion on this topic that was brought up several months ago.
That said, it's not entirely surprising that the Empire and Skyrim retain strong ties even at 4E 201. Cyrodiil's Colovian Estates are populated by a noble class that are descended from Nords, and have retained a "lite" version of Nordic culture and values. Tiber Septim, regardless of his actual ancestry, made a point of pretending to be Nordic (or, more technically, Atmoran; I suppose there's a difference, but the Nords would only respect him more for it) in order to garner Nord support for himself (hence the idea of Talos as a "native son" to Skyrim's traditionalists in the game). One of the game Skyrim's faults, in my opinion, is that the devs forgot just how strong the ties between native Nords and Imperials should be, considering that half of Imperial culture comes from Nordic roots (to say nothing of the foundation of their [Third] Empire in the first place). Skyrim and Cyrodiil should have had a very strong relationship, and while that relationship might have been strained by the signing of the White-Gold Concordat, the Empire should have been able to patch up those relations with skilled diplomacy (which, prior to Skyrim, they had been famed for)--but instead, the Empire is presented as snobbish and completely unfamiliar (and unconcerned) with Nordic culture, despite the fact that by rights that same culture should be part of the Imperial heritage! (The discussion between Proventus Avenicci and Hrongar takes the cake: Hrongar tells the story about Talos being called to High Hrothgar, and Proventus asks what this "Nord nonsense" has to do with the Dragonborn, ignoring completely that Hrongar's story was published in the first Pocket Guide to the Empire by Tiber Septim himself as proof that he was the legitimate ruler of Tamriel--in other words, this is not only part of the foundation of Proventus's Empire, but it was also published far and wide as propaganda by Tiber himself, so that Proventus must have heard it at one time or another.)
Long story short, the small number of passes between the two regions probably has less to do with Cyrodiil's continued control over Skyrim than the cultural and historical ties that Imperials and Nords share. Up until the White-Gold Concordat (and the aftermath, in which the Imperials apparently forgot whole sections of their own history for the sake of plot convenience), Skyrim had never had a reason to abandon the Empire; it has been considered the Empire's backbone since the Empire's foundation (even in the First Empire, Skyrim was not part of the Empire, but nonetheless Nordic support was crucial in founding the Empire under Alessia, and King Borgas of Skyrim was a fervent supporter of the Alessian Order in later years).