I got Fallout 3 3 days ago.

Awesome game.

I was wondering, if I left stuff somewhere like Moiras shop, whould it still be there when I came back, all the time?

I got some power armour, power helmet and fat boy and mini nukes today. Also a sledgehammer.

I had to drop the power armour and helmet, and sledehammer, though. I cannot use the power armour and helmet yet but I still wanted them.

I was overloaded.

If I dropped stuff at Moiras, would it still be there all the time if I left it if I was overloaded?

Anything you drop on the ground/floor will stay there forever with a couple of exceptions:

A hostile character will pick up a weapon that's better than the one they currently have (not usually a problem in Moira's).
A hungry character will swipe an item of food.

Until you get a home of your own, you might try using the Nuka Cola machine in Springvale for storage. It's a pretty safe place to stash your stuff.

Thank you. I have a house in Megaton now. So I am going to put any excess stuff in there.

I also recommend getting a companion that can be you walking storage cube (COMPANION CUBE <3), so you can give them stuff that's heavy but will get you a lot of caps to sell.