I fear that this would not be an RPG any longer. The point of an RPG is that you customize and develop your character, but that is rather impossible without leveling.
You could have extensive character creation, with a level cap of 1.
Personally, as much as I think XP systems are terrible for sandbox gameplay, I loathe learn-by use. It wouldn't be so bad if it was just attributes (that could easily be fairly invisible), but it tends to promote grind over fun.
If 'twere up to me, I'd change the system to this:
1) Use core quest XP for leveling; but restrict the impact of leveling on character development to allowing you to gain the use of new abilities, 1 per level. (I'd also follow a quest design philosophy that would replace the various isolated guild questlines with a few interrelated plots that any character could get involved with, but from differing perspectives based on their guild affiliations. On one playthrough, you might be guarding a merchant against assassins. On another, you might try to assassinate him. On a third, you might be aware of the assassination attempt and try to use it as a distraction to steal a valuable artifact from his house. On the fourth, you might be sent to negotiate the purchase of a relic that interests your magical institution, and get caught in the wrong place at the wrong time. So, rather than a bunch of random and unrelated events, each character's story would have a coherent narrative across all of their core quests, including but not limited to the main plot. But that's a little off topic).
2) Replace each skill with a list of related abilities (for example, replace Acrobatics with Basic Jump, Long Jump, Wall Jump, Wall Run, Vault Attack, etc. or replace a spell skill with the Spell Effects from that school). I wouldn't worry about obsessively balancing different archetypes too much, and boring skills like Heavy Armor and Light Armor would only get one or two abilities, while spell schools would get at least one for each spell effect, along with some other potential perks (I can envision advanced Conjuration specialists playing the game like Overlord, while someone with Acrobatics, Athletics, and Blade plays something like Prince of Persia).
3) You would unlock each ability for purchase through various plausible in-game means (minor racial/background/specialization bonus, paying a trainer, completing a quest, consorting with daedra, performing research, or some non-grindy sort of "practice" activity or mini-quest or achievement). You could unlock as many abilities as you want, but this doesn't provide any direct benefit. You would still need to spend an ability point (gained at level up) while resting to actually obtain the skill.
4) Attributes (Strength, Endurance, Speed, etc.) would improve through use, completely separately from XP or abilities. They would influence ability use in many ways, and using abilities would allow attributes to improve. However, even simple non-ability actions (like walking around carrying a hundred pounds worth of swag, or socially interacting with people without trying to manipulate or persuade them) would build up attributes. The speed of attribute improvement would decline as the attribute got higher, and also as the total of all attributes got higher. Not sure what to do about luck, but I imagine taking actions with fairly random outcomes would be more likely to improve it -- so, for instance, a spell that does 0-200 damage would improve luck, while one that does 100-100 damage (the same average damage value), would not.