After a few days of constant crashing I've got rid of the 60 mods I had installed.
Clean install of only the GOTY edition. (No mods or patches)
Win7, run setup as administrator on E:\Games\Morrowind
Played for 3 hours without a crash (1st time) But on exit I get the old "Morrowind has stopped working"

System:
Windows 7 SP1
2600K @ stock
580GTX latest drivers
16 gig ram

Morrowind.ini Default settings.

No other games, modern or old crash on exit.

Where do I start?

Its not really that serious is it. My game always used to do that on XP. Not on a win 7 though

My game likes to do that. I wouldn't worry about it. Your stuff is all saved, so I'd just let it crash.