IMHO, it
would work very well, although not all attributes should visualize themselves in the same way. For example, when you have high athletics your character may sometimes jog in place... that's all fine-and-dandy for your Arena Champion, but what about the Dark Knight who strikes fear into the hearts of citizens? Not so much.
Things should be represented in a more realistic manner. Characters with more Strength/Athletics would probably be more muscular, characters with more endurance would be a little wider, and maybe other things could visualize themselves in other ways like:
Destruction making your character create a ball of fire/frost/lighting in their hand and then get rid of it.
Acrobatics making your character jump in a different way.
Personality making your character acquire more compliments while passing through the streets.
A character with high sneak may gain new sneaking animations, looking stealthier than before.
All of those sort-of things could manifest themselves visually without making you go "UGH! That destroys my character completely!"
I mean, when you have high Athletics people comment on your muscles already, so it's not like you're changing much other than looking like you should. But that doesn't mean it should get taken to the extreme either and all of the characters should be running around with so much muscle that they can hardly move. Balance is key.