I agree with firefly, they should be less powerful but have like 20-40 guards depending on the size of the city.
If the towns are about as populated as the ones in Oblivion then 20-40 guards will be far too much. I want towns to feel like towns... not army encampments.