Once used, the locked door/chest should be unlocked forever, so if you are sure you got all you can with a Key, feel free to drop it.
Not true in general, but usually the case when a key is involved. Anything marked as a persistent reference in the CS has its state remembered in your save game, and most "key-only" doors/containers are set up that way, but it's not automatic, so don't count on it. There are several situations where a door or container has a key, but the lock is also pickable, and they're less likely to be persistent ones than the type that can't be picked.
There's also the special case of the Sigil Key in one of the Oblivion worlds. Since it's supposed to be a new, but identical, world each time you enter it, you need the key to open the door every time, and it may even be a new key as well (in the inventory of the Sigil Keeper).