This is pretty silly. It was the same way in Oblivion; I ended up just using Wyre Bash to virtually remove this "feature" of the game. Does anyone like seeing arrows sticking out of their character's body? I'd wager that most do not like this, or are at least indifferent on the matter. It's actually worse now in Skyrim, as there are bugs which make the arrows remain for long periods of time (I've had an arrow stuck in my character for days now). It's odd that I need to resort to console commands to fix this.

just give the arrows to your companion..al of them. and the arrows wil dissapear from you

Or at least have a cleanup script that puts decay timers on all cosmetic arrows at the end of combat.