The problem with removing skills, is That they are actually still there - in the background. Sounds good right?
Well not exactly. The problem is that in order to max certain 'skills' you instead have to max special to get the side benefit.
So if you want to pass all speech checks for example you need about an 11 or so in charisma, previously you just need a 100 in Speech.
Getting permanent 11 charisma isn't exactly easy, or recommended.
Now certainly there are consumables/wearables whatever to get you there. The point is certain things can be a lot harder to achieve now. My thing about speech is just one example.