Have you ever seen a Nord only 1.70 metres tall (5' 7")? That would be funny, but maybe realistic too, seeing as in real life just because 1 race is known to be tall, doesn't mean everyone is.
Everyone thinks of Scandinavian people as all being really tall, thats not the case, have the same average height has everyone else in Northern Europe, Germany, Britain etc. Pretty much the same average height as people in the U.S. in fact. Now average height is debatable mind you but generally average height is considered 5'8 to 5'10 (I give a bit of range here because there is a slight difference of a centimeter or two between some European countries and a lot of people can't agree on average height especially if you ask clothing manufacturers). Plenty of Scandinavians fall into this height range. So I agree that it's unrealistic that
every Nord, Imperial, Dunmer etc is the same height as everyone else in their race.