If the number is Red then it's probably either been Drained or Damaged. If it's Drained then you just have to wait for the effect to wear off. If it's been Damaged then a simple Restore Blunt spell/potion should return it to normal.
If the number is Black than you had an item with Fortify Blunt equiped and have since taken it off.
I checked all the skill levels and they appear to be blue in color. I don't ever recall seeing a restore blunt spell. Where might I find this spell, just in case? Thanks for the help