But you can! Ignore the "numbers" and you'll see that as far back as Morrowind certain attributes have been better than others regardless of the skills they govern.
And that's why Morrowind was so horribly unbalanced, because some choices were just better than others.
And by your logic about swimming and jumping, you could also argue the same for other skills. Being good at fighting with a one handed sword doesn't guarentee you can fight well with a two handed claymore, for example. Sure, both are bladed weapons, but both weapons are used differently, and when it comes to skills, what really matters is how the weapon is used, if you want to be entirely logical, not by what its edge looks like, in this respect, the approach Skyrim seems to be going for actually seems to make more sense as weapons are divided based on whether they're one handed or two handed, not whether they have blunt edges or bladed edges, and perks are used to allow further specialization, this way, using a claymore actually DOES require a different set of skills from using a one handed weapon that happens to also be a blade, but no one seems to care about that, probably because it's always been that way, whereas reducing the amount of skills is a change. In other words, what bothers players is not that the new design is bad, but that it's different, that's certainly the impression reading these forums can sometimes give.
In the end, no RPG setting can have a skill for every single individual thing you could reasonably expect to perform that might realistically require its own set of skills, for one thing, the more individual skills you have, the harder it becomes to ensure they're balanced and each one has its own merits, and when you start trying to make skills for everything, you get skills that are entirely pointless because what they do is so trivial that there's really no point in raising them. Sometimes, it works better to have things that might realistically require different skills, but for the purposes of gameplay, are connected enough to justify them being in the same skill.
In the end, I judge an RPG system by how well it actually plays, not how many skills it has. What's important is how much room it gives players for creating the character they want, and in that respect, the addition of perks could help a lot as it allows players to specialize their characters in ways that the basic skills don't allow. But some players don't seem to be able to see that, or choose not to, all they see is that there's less skills and act like it's the end of the world all of a sudden. Yet would they have complained if the series only had 18 skills from the start?
Now, I'm not saying reducing the amount of skills was a good idea, all that I'm saying is that less skills doesn't necessarily mean less role-playing, especially when perks are added in to the mix, as those are a factor past Elder Scrolls games never had. For better or worse, the presence of perks will change the gameplay experience.