Your block perks will only work when you are blocking, with a weapon or a shield. If sword and empty hand lets you block, then the perks will apply. Be careful for the perks that mention shield only though.
EDIT:
I will hijack the thread, hope OP won't mind.
Maybe someone will know. Are the perks shield bash/deadly bash/ disarm bash worth taking? I've got some perks free and I really don't know if it is worth taking those perks, or they are rather useless.
Shield bash = win. Dragon about to breathe in your face? shield bash. Two handed bandit about to power attack you? shield bash. Mage about to blast your face off? Shield Bash. as long as you have the stamina, shield bash is ridiculously good. Deadly bash, not so sure about. Extra damage is great and all, but thats not the point of the bash itself. Disarm, i dont have yet. planning on getting it though.