Trying to have a build where I don't branch off into unrelated skills is hard for me. I always want to do everything.
I never get to invest into the speech tree because I don't have the perks to spare (if I did I would be under-leveled and gimped most likely)
Lets say I want to do Illusion.
"oh but I should compliment it by using conjuration, never mind destruction which is pretty useless at certain levels"
"Oh and I guess I should take up alchemy because at some point everything becomes too high of a level for my spells to affect them"
"this is really starting to hurt, maybe I should take up alteration"
"maybe I need to do enchanting because all these schools of magic are costing me too much"
Or the fact that when Im a warrior I can never stick to 1h, 2h or archery for long. Its hard to stick to heavy or light armor since there are some that look so amazing.
Stuff like that. I have problems.