I've just read that armor protection is capped at 80% if your armor rating is 567. Do the Stoneflesh or Oakflesh spells increase this cap?

No.

The cap is absolute. You can boost your armor rating above 567, but anything above it won't count towards protection, outside of a handful of cases where an attack automatically bypasses X amount of armor rating.