I haven't had the problem with arrows yet, but I did have an ice shard stuck in my shoulder after fighting a mage. Regardless, I took
all the arrows that were in my inventory and put them in a container and closed it (I was in a first-person view at the time; I'm not sure if that matters). After doing that I switched to a third-person view and the ice shard was gone. Then I retrieved my arrows from the container and saved the game. See if that works for you.