But there is the ever present fear that the stuff might delete. I believe in the other games, once "too much" stuff has been done is when the game does "garbage collection" and deletes or looses track of items in containers that you do not own.
This won't happen. I have been leaving stuff in containers all over the map. It is always there when I come back.
Here is what happens with the containers. Some are static in their loot, some are leveled so that the loot is different the first time you open the container depending on the level you are at. However, once you open the container the first time, it sets itself so it will not change.
If the containers shows a Blue (or non red as I use a Blue HUD) so it does not indicate ownership by someone else, then it is safe for you to put stuff into. If you put something in a container that is owned, then the ownership of the item may transfer.
You can not put items into containers that are set to Respawn so it will not get lost that way. Someone might want to test the Legion Containers and I don't think the vending machines respawn, at least I have never seen one get more soda in it.
As for early game locations to store stuff, why don't you go take a loot at Victors House in Goodsprings. Then, you can rent a room in Novac for the entire game. Heck, if you do it right, the room is free.