At first I was thinking "not if you're starving", but then I figured that stealing is still a crime if you're poor/hungry/thirsty/unarmed (even though NPCs LITERALLY NEVER USE said items), murder is still bad if your target is an ass (MORIARTY I'm looking at YOU) and sleeping in untouched, yet owned, beds is completely disallowed.
Yes, it should still lower karma - it measures what choices you make, and under what conditions. IIf you only ever eat while absolutely starving, the occasional karma loss means nothing anyway (compared to someone who starves instead, who gets to die feeling noble).
Karma is intended to represent your overall choices so that individual characters can respond to the representation you've built up. If anything, karma changes should only occur with witnesses, like the Skyrim bounty. That way you can have a reputation of being a stand-up fella while having a basemant full of butchered raiders (and that one woman you murdered for her outfit).
It would also mean that being 'good' required more than simply not rocking the boat. People had to notice you were a well-to-do as you help them out with quests and such. (The downside is that it means stealing wouldn't affect your karma unless people observed it, under which conditions they would attack you.)