The UESP lore article on Redguards. At the end of the first paragraph it mentions that the Redguards do not come from Atmora like the other human races. They came from Yokuda, an island that was ravaged by a terrible disaster some time in the past. The Redguards also have very different views of the metaphysics of the universe they inhabit. A list of major Redguard dieties and/or cosmic forces can be found here.
The Redguards, or Yokudans, like all members of the human race, originated in Tamriel. Later, they left Tamriel for Atmora, to the north. Later still, many would leave Atmora, some returning to Tamriel and some venturing to Yokuda. These people became the Yokudans, who later left Yokuda after they destroyed most of their homeland with their nuclear-swords. These refugees fled to Tamriel, where they became known as Redguards, a corruption of the Yokudan words Ra Gada ('Great Warrior Wave', I think).