I got the idea after reading of High Rock nightblades in TES lore.
His build is a maxed out magicka and low stamina and uses alteration with robes instead of physical protection (armor).
He is already high conjuration and uses a bound weapon as his main attack.
He is already high in illusion and destruction.
So all I've had to do is change gameplay style and use DB robes instead of dressing like a mage.
I heavily rely on muffle and have started to use more sneak in the shadows and I conjur a bow or a dagger (ideal as I am low stamina so weight is not an issue)
What do you think?