To counteract this they could (and I'm just freeballing here) have some sort of race multiplier but no attributes. So like maybe the base damage is determined by what weapon you're using and what your skill with that type of weapon is (lets say the base damage of your particular mace is 30 pts) but then maybe races could have "unnacounted for" multipliers that apply to the base damage (maybe a nord has a 1.5 multiplier so a 30 damage mace does 45 damage and a wood elf has a .8 multiplier for 24 damage)
I don't think weapons work the way you think they work if you seriously think a wood elf is justified in doing half the damage a nord can do with the same sword. Skills using the sword matter a LOT more than the raw strength behind them, or at least it seems that the way Bethesda wants to build the TES world.