So Endurance, Luck and Agility are really the only ones that seem to constantly add direct benefit.
I kinda went that way, but Luck at 7 is probably overkill. Almost impossible to lose at blackjack.
I really wanted INT higher, as that is a good skill check option for Dead Money story/dialog. I also would go more INT next time for skill points. I went with STR too and really don't think it's all that great. I have too many Action Points and they regen fast because i took FO3 style perks early on. Level 30+ you simply don't need all that AP anymore.
AP was great when I was low level and fighting Deathclaws.