If I put Charisma down to 1 what would I lose out on
Charisma is useful for keeping your dialog options open although you can pimp your speech skills, charisma is not nearly so important as Intelligence. It determents the maximum number of followers and is equal to your CH divided by two. Even if you don't plan to keep any company you might want to set it to 2 so you can at least pick up NPCs temporarily for xp, carry stuff and such. It can be increased by 3 points during the game by gaining the charisma perk, surgery and with a specific item. It can also be decreased by certain special perks.
Another thing is that you cannot get certain perks, but that goes for all stats. See perks list at (minor spoilers): http://fallout.wikia.com/wiki/Fallout_2_perks
And you cannot sleep with certain npc's in Modoc and New Reno, if I remember correctly which has its advantages and disadvantages.
That's all I know the rest you'll have to look up in wiki or Per's walkthrough.