I think many people forget that "back doors" did not begin with Skyrim. They first appeared in Oblivion (Vilverin being a prominent example). But they used them more tastefully and sparingly in that game. Back doors didn't feel quite so shoved right into your face in Oblivion as they do in Skyrim. I think the problem in Skyrim is not that back doors exist, but that, for some people, they felt so monotonously regular. You could count on one being right there, exactly where you needed it, when you needed it, each time. I think it's the regularity that mainly gets on many people's nerves.
Bethesda has a habit being a little too over-enthusiastic with their new ideas. When they introduced scaling in Oblivion, for instance, they were way too heavy-handed with it. It was everywhere you went, and it was oppressive. But they learned their lesson and cut way back on scaling when they made Fallout 3. I think the same thing may happen with back doors. I think we may see a lot more variation in how back doors are used in Fallout 4.
As I said earlier, I like back doors. But I think it would have been better if they'd made a few Skyrim dungeons without back doors. And made the ones that were in the game a bit less glaringly obvious. I think doing just those two things would go a long ways towards making some people less annoyed with them.