In that situation, it's really not. How do you justify that with the Oblivion scenario? "When Baurus asked if I was a Nightblade, I said no and said I was a Bard, now I'm stuck being a Bard when I realize I don't really want to be one because my skills lie elsewhere..."
Doesn't really make any sense. Choices you make from a gameplay perspective like that don't really play in to roleplaying.
Well, you are really only stuck with having been a bard before going to prison, in this instance being told the world is at stake is a perfectly good reason for anyone to consider a change of career.
What I mean is say you take a perk to improve your sword fighting, then really soon find a cool magical mace, I would not reload an earlier save to change it to a mace perk, that would spoil the whole game, instead better to be stuck with some ability you might not use it's still a part of your character.