I think the best way is to make both the player skill and security skill matter more. For example, you can only start opening hard locks at level 25 and you can only start opening very hard locks at level 50. Then at level 75 your character can muffle the sound of a lockpick breaking, and at 100 lockpicks don't break anymore. Everything else (how much leeway you have with your timing, the chance of keeping several pins up when you fail etc.) is improved for every single skill level.
Then, the lockpicking should be done in real time, so you have to hurry up sometimes to pick the lock in time before a guard shows up, and as indicated, a lockpick breaking would make a loud enough sound to wake a sleeping person up if you try to open the jewelry box on the nightstand at least.
The lockpicking minigame in oblivion really had the same fate as the speechcraft minigame. You can take all the time in the world, and it's not so bad if it goes wrong. This meant there was no challenge, and no reason to improve your security/speechcraft skill.
if bashing locks has a downside.........what exactly is supposed to be the downside of having alterations spells? there arent any downsides, making lockpicking completely useless to pick as a skill. might as well take that skill away as well. if alteration only went up to normal or hard locks and couldnt do very hard locks or if there were certain locks that couldnt be opened with a spell, that would make picking lockpick viable again. this is the same issue with healing/cure disease potions versus their magic equivalents. you get both sets of spells super early on rendering potions entirely useless for the rest of the game.....unless you have heavily mod the game like i did with super slow magicka regen.
Well, first of all the "open" spells with alteration requires a lot of alteration skill. It takes journeyman level to open average locks after all. Of course if you powerleveled and had alteration as minor skill this isn't a problem, but that's not how most people play the game, and that issue won't be present in Skyrim. Secondly the alteration spells could be made loud. So it would be useful for opening loot in caves, or in basemants of people's houses, but not for house doors or anything in a room with an NPC present.