This same thing happened to me. Tried to test it by adding and removing perks with the console commands but the results i was getting were unconclusive with the exception of the one undeiable fact. The damage displayed resets to unbuffed stats. As far as damage that is actually being dealt I haven't been able to find anything useful out.