OBSE's SetRaceAlias allows you to declare that a custom race will be treated by the game as one of the standard races
if you use SetRaceAlias
Customrace Dunmer then anyone of the Customrace will have GetIsRace Dunmer return true, so scripts, dialog conditions etc will work, in vanilla or mod-added code.
I don't recall anything In Wrye Bash doing it externally. That function would have to be called in a quest script each time the game is loaded.