You can actually reinstall them, just go into the mod screen for the appropriate weapon, and instead of "build" it should say "attach mod" or something like that. You can also scrap them by dropping them in a settlement and then pressing "scrap" in the workshop menu.