You need to take the first skill in a skill tree to start unlocking skills further up that tree. Assuming you want to wind up with skills farther up the tree, that's one point per skill point. Assuming you use skills from six trees and want a higher up skill, that's 6 points for prereqs.

With weapon swap, as pointed out, you'll have 12 active skill slots. Each of those 12 skills has a morph which costs a skill point and basically transforms it into a different (better) skill. So 24 points there (skill + morph).

Then there are passives. Lots and lots of useful passives, which take multiple points each.

And crafting.

And some situational skills, like soultrap for filling soulgems (don't need it slotted all the time) and synergy skills for group instances.

So, yeah. You need about 28-30 points for your active skills, a handful for situational skills, and whatever you want to spend for crafting; the rest go into passives, which will easily absorb all available skill points like a sponge

As for WHY you only have 12 slots, the answer the devs give is that it's a deck-building game *shrug*. Lot more skill slots than the single player TES games give you: left and right click, swap manually if you want anything else just like ESO.