Yeah, that's the issue. Like you said, in Skyrim, by my second playthrough, I could pick ANY lock with only a skill level of 5 in Lockpicking. I'd just break a couple of lockpicks on the harder difficulty locks before finding the sweet spot.
I'm really not sure WHAT could be done to completely fix the system. Like others have said, the way Fallout 3 did it, by making you have 25, 50, or 75 Skill, etc. for the different lock levels worked okay, but it made any other points in that skill useless. But from what we saw with Skyrim, Bethesda wasn't entirely happy with that lockpicking system, so they tweaked it to let you TRY any lock, regardless of level. Your skill just made the correct "zones" on the lock larger and easier to find. But with enough personal experience with the lock system, you could baby and feather open any lock if you were willing to sacrifice a few lockpicks.
One solution I can think of is this:
1) Increase the difficulty of all locks. So the zone on a Very Hard lock is an almost impossible couple of pixels for a character that doesn't have the Lockpicking Perk.
2) Drastically reduce the number of bobby pins / lockpicks available. Make them hard and rare to come by, but more durable. The Perk system could be set up so that if picking locks you are trained for, you'll almost never break a bobby pin, it's just a matter of patience. Make the likely-hood of breaking a bobby pin jump up considerably with locks higher than your attained Perk levels. This way, opening a Very Hard lock is possible if you are willing to sacrifice a lot of bobby pins - but with the scarcity of them, it will very likely be impossible to do this every time. You won't be able to continuously open locks above your Perk level, but you can open a few of them if willing to sacrifice a hard-to-get resource like the bobby pins.
3) Solve the "limited bobby pins" problem for players dedicated to Lockpicking by making the final Perk level grant them a skeleton key ability where they get a bobby pin that never breaks.
The hardest part of this system would be balancing the number of bobby pins in an open-world, and communicating to the player that, "Hey! These are a limited resource not easily replenished!"