You can only join one house at a time. Each House has a different storyline, favored skills, and orientation.
House Redoran - Honor and Duty, favored skills are for warrior characters. They tend to have a lot of missions to defend your house's honor, defend the weak, avenge your fallen brothers, etc. Their rewards are probably the worst, but I enjoy the House alot.
House Hlaalu - Greed and Capitalism, favored skills are more for rogue/thief characters. A lot of these quests are shaking people down for money, intimidating people, acting like a criminal. I don't think I've ever finished this house, it doesn't appeal to me much but they do have have the most rewards in terms of sheer gold offered.
House Telvanni - In my (and most people's) opinion the best house, they are isolationist wizards who favor skills for wizard characters. Their missions are pretty broad, ranging from murder, recovering artifacts, political intrigue and more. All of the important characters of this faction live in wizard towers and hate outsiders, but the wizard towers don't have stairs so you need to be able to use a levitate spell to get up and talk to people, which is kind of an interesting quirk. Also what's cool is their laissez faire attitude towards how you interact with other House members
If you steal from another member of House Telvanni and live then clearly you deserved what you took. If you kill another member of House Telvanni in a dispute, then clearly your argument has more merit meaning it's basically impossible to get kicked out and you can really run amok. Telvanni rewards are pretty awesome too, though most of them are free spells, or enchanted items with useful/cool effects, as opposed to just getting big bags of gold like House Hlaalu or a 'warm fuzzy feeling' with House Redoran.
Really which Great House you join is determined entirely by the type of character you play, as a heavy armor wearer who uses swords and no magic won't be able to advance in House Telvanni, and vice versa. If you don't like the House that fits your character's archetype then you join one of the other guilds themed towards your skill set.