I won't discuss Magic, because that has quantitative measures of difficulty (some spells use more magicka, do more damage) whereas weapons are fairly static.
And while I agree that "you cannot master Blade by chopping onions" very few people will be killing rats to get to level 100, the nature of the grind actually makes that "harder" to do because it's boring.
The system already has it in place so that let's say, to go from level 1 to level 2, you need to hit an enemy 10 times. Now, to go from level 2 to 3, you need to hit an enemy 20 times. Now, this has a twofold effect. At first, it is hard to level because you skill is so low that you rarely connect hits. At a mid level, you level quickly because you are hitting often, but still making skill gains in small increments. At the highest levels, you are once again slow to gain more skill because although you are accurate with your attacks, it takes many successful attacks per skill gain.
So even though there's no measurable difference between an attack at level 1 against a rat and an attack at level 25 against a troll and an attack at level 75 against a frost giant, you still make different skill gains each time. Yes, you could sit there killing rat at level 25 and eventually get to 75, but few people will do that because it will take so long. By design, as soon as you are skillfull enough to kill harder enemies without dying, you will. And thus increase your skill further.