I like the idea of rival guilds.. I would think the rival guilds would be closer related though.. Thieves guild or assassins guild. Fighters guild or mercenary guild. Guilds that are actually in competition with each other and therefore know each others members. Thieves guild might know who's in the Imperial Guard but the Imperial Guard wouldn't know who's in the thieves guild (otherwise they'd just arrest them.) It would be more likely with opposite guilds you could go up to a certain point in the quest line and then have to make a decision.. Such as you work for the Guard and have to infiltrate the thieves guild... you then get to do most of the thieves guild quest line and guard quest line until it comes down to a decision that would take down one or the other.. so only allowing you a master class in one or the other.
While I agree with _LPS worry that simple rivalries between guilds is a little too cheap and bland, I think the suggestion made in the latter part of this post is a good one. I don't think joining one faction should automatically forbid you from joining a faction with somewhat opposing interests. Rather, if you have sufficiently low notoriety, you should be able to join both, but then, just to borrow your example, as you progress in the Imperial Guard, there comes a point where you can longer get any further in the Thieves Guild - indeed, you might be forced to choose which guild you want to remain in.
The infiltration idea is also a good one. But I suspect that's slightly easier to achieve on a quest-by-quest basis. That is, instead of joining the Guard, and then, in order to infiltrate the Thieves Guild you become a member of that faction, it could just be a quest of the Guard where you need to become friendly with some members of the Thieves guild to gain information or whatever.