I've been through 2 playthroughs and everytine I goto the crimson caravan camp he's no where in sight... Is he in one of the buildings? Or do you need to do a certain quest

He should just be standing outside by his stall ready to sell you stuff.

my first playthrough, i didn't even realize there was supposed to BE a merchant at the Crimson Caravan place (although i thought it was kind of weird there wasn't) because he just never appeared in that file. sometimes the caravaneers get stuck together somewhere in the Crimson Caravan area, either in one of the buildings or somewhere outside and he can get stuck with them. this has happened to me a couple times in other files, but recently he just disappeared altogether only to reappear after several saves/reloads. at this point i don't even try to rationalize the weird [censored] this game does

Go into the workers' barracks. Sometimes, the actors' (the NPCs and creatures, namely) scripting will not trigger until you actually enter the same cell or section of the map in which they load up. You should likely find him in there, and he should head outside immediately after you enter the barracks, and then take up position at his vendor stall, from which he likely never will move.