probably because I'm one of the few people who really care about those at all
That isn't why I talked about the Imperial City. I brought up the Imperial City because I didn't realize that when you used the word "town" you literally wanted to discuss small towns. I hate to say this about people here, but most posters on these forums use words so inaccurately that, over the years, I've kind of abandoned any attempt to read posts literally. I just assumed you meant any kind of settlement where people live. My apologies.
So if we're talking about smaller towns, I'd say my biggest disappointment was Weye. It seeems to me that, being so close to the Imperial City, it ought to be larger, more bustling. It's kind of the gateway town to the Imperial City and should be raking in some serious coin from travelers stopping there to clean up and refresh themselves before they enter The Big City. For me, Weye was a missed opportunity and a serious disappointment.