My follower uses a bow that isnt in his actual inventory. I did removeallitems and he gave everything, but his bow.

I checked him out in the creation kit, he doesn't have one in his character config.

I really do not want my follower using a bow.

How do I get rid of this phantom bow?

I use a follower MOD to manage follower items and behavior, using http://www.nexusmods.com/skyrim/mods/15524/? right now, but there are many others I haven't tried yet. EFF and UFO come to mind.

The default bow re-spawns multiple copies in the vanilla game if I am not mistaken. Maybe check USEP Followers page, specifically the equipment section for more details.

Bow and arrows are added to the followers in their follower quest alias.

To be brutally honest, what is missing from Vanilla is the Melee / Ranged options in Follower conversations. Why they designed these into the Vanilla game in FO3 and not Skyrim is a mystery.