I feel that. Beyond the lack of summon variety in Skyrim, my biggest beef with Conjuration in that game was the vertical progression of summons. Flame Atronach, then Frost Atronach, then Storm Atronach, then Dremora - it wasn't as bad, since you could bring all summons up to par with a perk and they all did different things in combat, but as you progressed the skill it felt like there was no reason to use the earlier summons you got now that you have the better one. I do really like Skyrim's system of reanimating corpses, though, instead of just summoning undead creatures.
I'm really hoping they bring back spellcrafting, and model it more on Fallout's weapon modding instead of the system used in Morrowind and Oblivion. By that I mean, I don't want fine control of variables like Magnitude, Area, and Duration nearly as much as I want just a greater variety of options to mix and match, and putting set magnitudes behind tiered perks is better for balance anyway. Hell, Daggerfall was better, since you could make spells that scale with your level and choose area-around-caster as a delivery type. So for instance, knowing Wall of Flames and Frost Rune lets us mix & match flame damage, frost damage, streaming casting, rune casting, and the lingering damage well effect that the Wall of Flames leaves; so maybe we could use that to create a Flame Rune that leaves a lingering pool of Fire Damage after it gets set off, or a basic Frostbite spell.
This is slightly related, one of my biggest problems with playing a mage in Elder Scrolls is that they have to join a mage faction to really excel - warriors don't need to join a group to craft/find the best weapons and armor, thieves don't need to join a group to move in the shadows and steal (Skyrim even improved this by giving us a Speech perk to sell stolen goods to any merchant), but mages have to join a group to get access to the best spells, or spellmaking in Oblivion. My mages want autonomy, dammit! With spellcrafting, or some means of learning new spells from old ones, I'd only have to find basic spells (either from a faction, unaffiliated merchants, or loot) and I'd be able to expand on them myself.