Why oh why did bethesda make so many essential characters for skyrim. Was it really necessary?

Short answer: no.
A longer answer would talk about casual gamers and the influence it has on the industry and having to make it accessable to the lowest common denominator. It is what it is though and I'm sure there is a way to fix it on pc

I wish they would make it like it was in Morrowind. Kill any NPC and drop any quest item. Just be prepared to suffer the consequences for such actions.