So it turns out that I like some of the agility abilities, but I'm about 4 hours into the game, I only set my agility to 2 in the beginning does this mean I can only choose the first 2 perks in agility? If so is there anything I can do to change this
It's not locked in that way. You have access to whatever perk you want so long as you meet the requirements. You don't have to have the first perk in agility to get the second one