When you marry a follower they get added TO a couple factions that are hidden, but they shouldn't get removed from any that they're already in.. except the one that defines them as a resident of the town or house that they currently live in (if you choose to live elsewhere). There are many eligible http://en.uesp.net/wiki/Skyrim:Marriage#NPCs_available_to_be_married that you can pick from, some of them are members of large factions and some have almost no ties at all. Marrying one should really only change what bed(s) they call theirs, and what house they live in and where they hang out when not following you.