Are you sure about this? I heard that everything in the perk tree applied to daggers, the only thing that doesn't is a certain 1h weapon damage enchant I thought
Relatively sure. I saved before a boss fight and sunk three points into Bladesman. Fought it with daggers. Reloaded, tried again, without the perk. Repeated this a few times, and subjectively there was no difference. Then I did the same with swords and damage output with the perk was noticeably higher. There have been some threads with more details, which concluded the same. This isn't 100% and it's really just my observation, but if accurate, it's not surprising since the description of Bladesman does say it is for swords only (and maces/axes have their respective perks). I feel it also makes sense as daggers instead get the backstab perk in the sneak tree.
The 100% extra damage with 1H weapons does apply to daggers, though. That's also much easier to test.