You can reverse pickpocket paralysis poisons on any NPC's and steal everything they have in their inventory without getting detected and zero chance of failure (excluded equipped items unless you have the master perk). You can also Ice Breath freeze them and do the same since its considered a non-hostile shout. If you were so inclined you could buy 20k in training for a high level skill, reverse pickpocket/shout and steal it back, or just regular thieving. Since pick pocketing is insanely easy to level this gives you access to a lot of fun gear you'd normally either pay thousands for, or have to wait until mid-late game to get. Master robes, enchanted weapons etc. You can master pick pocketing in short order just in Whiterun/Riverwood.
You can enchant a weapon with absorb stamina 1 and have infinite power attacks as early as level one, provided you find the weapon (if you dual wield, keep your dagger unenchanted and use elemental fury for lightning quick power attacks). A guaranteed stamina absorption weapon is located in Labrynthian.
Ebony armor and weapons are available at level one in any high altitude (harder difficulty) Nord Tombs, most deathlords can be killed with fancy footwork or stealth and a bow very early with patience, yielding ebony gear without smithing and needing to level to 35+.
When leveling block you can get almost a level per attack by blocking and partially dodging a giants power attack. This can effectively be done with the slow time perk; you can skyrocket blocking to 100 with a giant trying to squash you

.
Listening to bandit dialog while hidden is hilarious, also listening to Babette in the DB is hilarious.