I've done stuff like that before. What I usually do is:
1) Copy the background to a new layer, and put the logo layer between the original background layer and the background copy layer.
2) Then on the background copy layer, Color to Alpha/white, so that all that remains is the dark material detail, such as fabric weave and folds or wood grain, etc...
3) You may want to desaturate the copy layer at this point, or at least the portion that sits over the logo, so it doesn't tint the logo
4) And also lighten it and/or adjust the opacity or contrast of the copy layer so it doesn't make the detail too heavy over the original background. You can even erase around where it overlays the original background, leaving only enough to cover the logo, if that helps.
5) And then use the original normal map for the background without the logo, so the lighting treats the logo as if it were just another part of the material.
Good luck!