What annoys me is that, even though I am infamous with the Powder Gangers and Caesers Legion.. If I "steal" an item from either of them, I lose Karma.. WTF???

Karma really has no effect in Fallout anymore, its all Rep. And stealing only penalizes a miniscule amount of karma loss.
Like in the Silver Rush, everything not in a container or on a body i "carried" in to the back room and stole when i was hidden.