Will my saves be there after I reinstall my Windows? (and of course wipe C disk)
Fallout 3 uses Games For Windows Live which, as far as I heard, denies you from loading game saves after reinstalling Windows.
I have a Steam version of Fallout 3 G.O.T.Y. which might impact it.

How will it work?

The Steam version of Fallout 3 will only use GFWL if it's active and you're using "Online Mode" with Steam. You can happily play without GFWL and "Offline Mode" if you wish to do so. If you don't care for achievements you can completely ignore GFWL. I to have Steam GOTY edition and I prefer to just play offline and even use a mod to disable GFWL without problems.

Either way, your saves will still be found under "Documents\My Games\Fallout3\Saves" by default and yes you will lose them if wipe your hard drive.

I do care for the achievements. I want to use everything that is there to offer by everything I own, in this case - Fallout 3.
I heard that even if you make a copy of them - you won't be able to use them because of GFWL's special number which changes every time you install it.

Just copy the saves out of the current GFWL ID# and when you launch the game on the new install it will make a new number. Just move the saves back into the new ID number. That is how I deal with some of the GFWL games I have.

I thought it put the numbers on the save names too.
Anyway, thanks for the advice; where exactly can I find that folder with all the saves/numbers/etc. (everything that I have to copy in order to save my GFWL account)?

Normally the saves are stored in My Documents>My Games>Fallout 3>Saves
Though that may be different with GFWL enabled. I would check there first.