I'm sure I have seen setting where was time in second for how long it takes arrows to disappear.
Do you use Wrye Bash? If so, you can set this to whatever you want in the 'Tweak Settings' section of the bashed patch.
If you don't, its probably not worth it to install and learn it just for this feature (but if you're modding regularly, its highly recommended). Just depends on how keen you are.
As far as I'm aware there are no Oblivion.ini settings that can do this.