As you walk straight into red rock cavern there is a path a the back that seems to go up into the mountains and offer a second way out (as in not having to simply turn around and exit the way you came in). But when I tried to walk the path i quickly realized it was too steep and I couldn't make it up.

Am I missing something?

You can get up there if you try hard enough but Cazador Canyon as I like to call it doesn't have much but death waiting for you.

You can also get there from the Jacobstown side. Just head to the Remnants Bunker, go straight to the mountain border and follow it to Jacobstown. Eventually you will see Cazadores and a path leading to their nest.