Penny in the settlement that you start the quest "Human Error" won't trade with me, she keeps saying "Enjoy your stay!" over and over and over...

I don't even get a dialogue wheel.

I had this. Try sleeping in the guest house overnight, It seemed to reset her script for the day and I could talk to her properly then.

I don't have to finish the quest first right? trading with her is possible before doing anything with the quest other than starting it?

I didn't need to finish the quest first, just slept anyways and as I said, got her talking rather than just making that greeting over and over.