Well, did you talk negatively with Father? Which division heads did you meet? You had to at least meet Dr. Li, since she installs the fast travel location for the Institute in your pip boy.
Without knowing what conversations you had, it's really hard to say, why they attack you, but you must have done something, either that or it's a bug.
http://fallout.wikia.com/wiki/Institutionalized
And not having a previous save is really bad. I have 100's of saves, always a quick save and an auto save... I don't know how anyone can play this game without saving it ever so often...