There are none. The only thing "custon" you can do to the anti material rifle are the bullets (i forget the name)
.50 Match Hand Load is the "named" bullet. You need "Hand Loader" (I think thats the name) perk to unlock the receipt to make it at the Reload Bench.
There are the normally avaliable bullets like Incendiary and Armor Piercing. Which are available at shops once you reach a certain level. Not sure which level exactly, but I saw AP fairly early.