I went to the head paladin fella and he said he'd use the stuff I found out to overthrow the current elder, asking for a few days.
After about two in game weeks I remembered the BoS and went back to see how things were, there was a patrol of 4 paladins on the surface but when I went to the elder it was still the original. Neither him nor the head paladin would talk to me, their out of conversation dialogue asking for a little time.
I waited three in game days at Novac then went back, no change, have I missed something or is it just another bug?