The way I play my Orc is by practically ONLY using charge up attacks (Not sure if its a RP thing or just preference). Does this mean I'm getting a lower amount of exp as compared to swinging it full speed?

I'm very sure the exp works per hit, not per damage. Which would mean charge attacks make it so you level up slower.

Someone correct me if I'm wrong.

Your blade (or blunt) skill will go up based on the number of swings, not the type or speed of your swings. And of course, the swings have to impact a living (okay, or undead sort of living) target.

Ugh, probably missing out on a few levels the way I play, guess ill find someone to teach me a few blunt levels to make up for it.

http://www.uesp.net/wiki/Oblivion:Increasing_Skills tells you exactly how each skill works on exp, and how much of a specific action you need to level the skill up to 100.