First, remember this is a role playing game. Your character initially has been a civilian for a while then in cold sleep for 200 years, so you cannot expect to be very good at combat right out of Vault 111. Even a low Intelligence character would realize that it is critical to get into shape, gather intel about this new, dangerous world, etc before charging out into the wastes. If not, the result will be pretty much what you experienced, so it works as a warning that the former activity is needed first.
Higher Luck is strongly advised. Your Agility doesn't need to be so high to start but you can do it that way if you want. Higher Perception too, but you don't have to worry about max.
Look at the SPECIAL and perks important for stealth. I'll outline some more detailed suggestions than what I said above as I always play stealth (not theft, but stealth).
Agility, sure, but 10? Not really needed, not at the start and not for a long while, really. 5 would be fine (and it's what I used) although you'll want to eventually get up to 7 for Ninja.
No need for Strength or Endurance as stealth is all about avoiding combat or eliminating hostiles before they notice you (or even avoiding them after they notice you). Don't need the perks in Strength, not even Armorer, not for stealth, and really don't need the perks in Endurance either (and don't need health as you want to avoid combat with stealth and there is plenty of healing available).
Perception is important but doesn't need to be high to start, same as Agility (see above). 5 is fine and can easily by bumped to 6 or 7 near the start of the game (no spoilers as to how but you may already know, of course, and if you don't, you will very soon by simply playing). You want Rifleman, obviously, unless you want to stick with pistols (the latter means take Gunslinger but be aware that pistols as a primary weapon do less damage so don't really fit stealth as well... still use them, but not with perks, at least not for a long time). Automatics are unnecessary and not really good for stealth, either, so don't worry about Commando. Both Gunslinger and Commando are under Agility while Rifleman is under Perception, but hardly needs any Perception at all to take so it doesn't matter as you want decent Perception to start, anyway.
Note that Perception perks also include Demolition Expert for explosives. These are fairly important and useful for stealth characters as an alternate, high damage means of taking out difficult enemies/targets. This includes setting explosive traps (mines) and luring enemies to them via stealthy behavior. This is a "later on" perk for the most part but worth noting.
The same note is true for Sniper, a very high Perception perk. Not needed for a long time, but you'll want it eventually.
One more note for Perception that also relates to Intelligence (below): Perception includes the Night Person perk that gives +2 to Int and Per from 6 PM to 6 AM (and +3 to both during that time at Rank 2 plus night vision while sneaking). This is an excellent perk for stealth characters as long as you build for it (i.e., no need to max Int or Per for a long time and certainly not to start since you will get the +2 or +3).
Intelligence is needed for both Gun Nut (ballistic weapons) and Science (energy weapons). However, note that Science is also needed for some other things such as Water Purifiers. Gun Nut is low Intelligence (3) but Science is mid-level (6). Intelligence also affects how much XP you gain for anything, so having a bit higher is helpful. It's up to you where to put the misc points but Int is definitely one option for stealth.
Luck is critical (pun intended). You want a decent Luck to start but don't need really high Luck at the beginning unless you want to put misc points there (see the note in Intelligence above about putting misc points in certain SPECIALs). However, Luck is focused on using VATS (key for stealth) and critical hit meter building and damage (key for stealth) as well as loot (somewhat oriented towards stealth). I start with 4-6, but I also always start with 6 Int (see next note).
Note that Luck includes Idiot Savant which randomly multiplies XP gain with a higher chance of activating with lower Intelligence. Even if you take lower Intelligence, you can choose to take mid-level Luck (5) and take Idiot Savant to make up lost XP from the lower Int. It's an option for balancing XP concerns with these two SPECIAL choices.
Charisma is useful but not essential for stealth. Depends on role play, really. On the other hand, it is also about avoiding combat which is key for stealth, so you may want to choose it for misc points. Depends on role play, as I said (i.e., how you want the character to be).
One important caveat about Charisma: it is key for settlements, at least mid-level (6) for Local Leader. Low level helps with loot (caps, Cap Collector) but is so low level it's not a worry. Local Leader is, though, if you want to do settlements at all (and I highly advise doing them, myself).
Obviously, Charisma also includes getting others to deal with issues (Animal Friend, Wasteland Whisperer, and Intimidation) as well as persuasion in dialogue, and influencing opposite six more easily (or eliminating them). These depend on role play, of course.
Finally, one more note: you do not need to worry too much about XP. Areas respawn including junk as well as locked containers, so gaining XP, even with low Int and/or no Night Person perk, will still be possible via looting areas repeatedly (and the respawn time is quite short in this game, too, even perhaps too short in many cases).