So someone who has never cast a destruction spell will be able to use powerful spells? So if I never dual wielded swords and equip two will I be able to do damage with it on par with someone who leveled up their dual wielding?
Swords require swinging.
A staff doesn't require knowledge of magic to use. It only has a limited number of uses (until you have to recharge it), so it's basically like a scroll that you can use a bunch.
I'm really not sure why you have a problem with it.