Well, they don't start with the Amulet of Kings in the vanilla game, it's given to them via a script (using Player.AddItem) when appropriate. But they do start with jail shirt and pants, which are in the inventory of the default NPC form. So you have several ways to provide things at the start: put them in the character's inventory, place them in the start location where they can be picked up, or have a quest script give them the item, and possibly others.
There are a number of alternative start mods you might want to look at in the CS to see how they did it.