I've played Very Hard. It's, well not very hard. Maybe not easy and it's maybe challenging but I doubt you need to worry about it too much. And I'm certainly not a FPS expert but was still able to survive combat well enough with Very Hard. As mentioned above, Very Hard can also be very unrealistic.
What is very hard is the game with FWE and MMM on normal difficulty. Those are great mods. And they definitely increase the difficulty. Your character can easily can be killed in under once second if you're not careful. PC only, but if you've got a PC I highly recommend those mods.
Fallout 3's "Very Hard" is another game's "Normal". This game needs "ulta super duper insane HC" difficulty level.
I could buy that. The last game I played before returning to FO3 is Mass Effect 2. Seems Mass Effect 2 on Normal was about as hard as Fallout 3 on Very Hard. Maybe ME2/Normal was a bit harder than than FO3/VH, at least on the first playthough (got easier the second time around).
The Ultra Super Duper Insane hardcoe is FWE, I think