I mean, we all know that the cities in Morrowind and Oblivion were all small, with small populations and also reduced in size.

Daggerfall and maybe Arena (which I never played) represent the size of cities in Tamriel much more accurately. Too bad they have very old graphics.

Anyway, does the official lore (be it Arena or Daggerfall) or books based on the series, ever say the real size of the Imperial City? What is it population, how big should that "circle" actually be?

It am interested in starting a mod that will just try to recreate, graphically, the Imperial City as it was supposed to be if Oblivion had a real scale and not a whole province at only 4km diameter.

I did a search over at the IL and couldn't find anything in regards to the population or how large the city actually is. I think you can only go on theory or inspiration.

I'd estimate that the Imperial City has at least a population of over or near 500,000. That's a bit of theory and inspiration there, usually when I think of the Imperial City, both Constantinople and Rome come to mind. Rome had a population of 450,000 by the year 100 AD and Constantinople had 400,000 in 1000 AD. Also considering the world, with magic and healing potions that work. The health and birth rates would be higher than Europe during that period. The Imperial City, while being populated mostly by Imperials, possess a diverse population.

As for the size, I hadn't gave that much thought. I just super sized everything with Tamriel. The closest examples of size I can think of is Venice and Tenochtitlan (The original Mexico City), Tenochtitlan a tad more largely because that was built upon a Island much like the Imperial city, minus the hill formation.

I hope this provides some help for you.