My friends were saying that Dwemer are short little Dwarves, but I was under the impression that they were the same height as the average human is?
No. As much as I worship Tolkien, LOTR (and WoW) have done TES a great disservice. Contrary to popular belief, the Dwemer were
NOT dwarves. They're about the same physical stature as their Chimer/Dumer and Altmer cousins. Especially according to the lore here:
Appearance
As previously mentioned, the Dwemer were no shorter than any other elven race, despite being classically described as such. Their physical appearance is hard to judge, as http://www.uesp.net/wiki/Lore:Yagrum_Bagarn lives bloated and deformed by http://www.uesp.net/wiki/Lore:Corprus, and the only other visual examples come from sculptures and http://www.uesp.net/wiki/Morrowind:Undead#Dwarven_Spectres found in their ruined dwellings. It can be deduced, however, that long beards were popular amongst the lost elves, and they preferred heavy metal armors and robes. Their remaining features could be described similarly to that of the other elven races, as no specific information exists....
http://www.uesp.net/wiki/Lore:Dwemer
and this is also supported by this wiki entry here:
The Dwemer are an ancient race of evles that are commonly mistaken as dwarves. Within Skyrim, there has been no proof of the presence of Dwarves, thus the Dwemer where in place.
....
http://skyrim.neoseeker.com/wiki/Dwemer
As an in game example, here's a comparision of Dwemer (left) and Altmer (right) Elven armor.
http://i770.photobucket.com/albums/xx346/watersprite2008/DwemervAltmer.jpg
Then again, these armor suits were put on 2 of my mannequins, so they might be the same height by default. My Nord never had any issues wearing Dwemer armor. But then again I suppose an Orc or Bosmer character could equally fit into the same suit of armor as well.

So perhaps the best way to tell is compare actual racial physical sizes. The Bosmer are the shortest and smallest of both Man and Mer species, while Altmer are technically the tallest. Try to verify this next time visually while in game. I had a hard time doing this since I've only played a Nord so far, and haven't encountered any Bosmers yet.

Besides, Nords are also tall, so it kind of hard to judge by looking at any of the Thalmor.