That is a pretty big investment of my time but there is one slight thing that bothers me. At the start of the game for example, during my first few levels I used destruction magic and assigned 4 of my hard earned perks to the destruction skill tree. I have since totally abandoned destruction magic and have settled into a heavy armour warrior type character.
I realise now that there are only 50 perks available for me to pick out of 250 and it looks like I have wasted some. I wish there was some way to reassign perks, even if it cost me a ton of gold or a very difficult quest, because I can't go back and start the game again at this stage.