During the High Middle Ages, yes. The Early Middle ages, however, was a horrible place to live (in most of Europe). Few cities, petty kingdoms, bad sanitation, nations were very, very poor, ect.
Do the Carolingians ring any bell?
As for nations being poor, they had enough cash to throw it at the Vikings on more than one occasion, deciding that was less of a loss than having them burn a town or engaging them in battle. The High Middle Ages didn't arrive from nothing. They build upon earlier developments like anyone else. In fact, a lot of those you call "petty kingdoms" (some of them covering territories larger than today's nations) were engaged in an "imitatio imperii", i.e. they were striving to be just like the roman empire. Some even worked towards what they called a "restauratio imperii", a restoration of the empire, obviously with them at the top. Which in the end led to Charlemagne having himself crowned emperor, and some successors later that empire being called "(Holy) Roman Empire".
What people forget is that Rome itself had long been a shadow of its former self, lodged in civil wars. Some of the cities in Germania were actually in a better shape, coming from the fact that e.g. Trier had been a capitol of the empire for some time. The roman infrastructure in Cologne was still used by the Merovingians. In fact, the infrastructure remained largely standing until the Vikings burnt down the city in the 9th century, the same fate that befell Trier. However, some parts of the roman structures there survived even later and the imperial baths in Trier were later integrated into the city fortifications when the Roman line of fortifications had become indefensible for lack of citizens. The water supply was still working, as evidenced by weavers' street and tanners' street in the vicinity.