I like to role play a law-abiding, good PC, but that's not always easy in Skyrim. One source of guilt has been killing giants for their toes and mammoths for their souls; they are utterly peaceful around their camps unless provoked (so don't provoke them). Of course, marauding giants are a different issue; I consider them fair game. So here's the workaround - maybe it'll help you RP.
Buy giant toes from vendors and pretend they acquired the toes honorably by way of permission to participate in some sort of secret burial ceremony to which only select outsiders are invited by the giants.
Leave peaceful mammoths alone, don't provoke them into fighting. Get grand souls by fighting Draugr and Falmer uber baddies. Personally, I don't buy filled soul gems but that's also an option.
Anyone else have workarounds for some of the ... evil... parts of the game? (I generally avoid the Daedric quests and the Dark Brotherhood. The Thieve's Guild is acceptable, though barely).