In terms of aesthetics, I like the silver swords the best.......but you can't upgrade them.

Glass weapons are my favorite (especially the glass sword and greatsword), I use ebony dagger and ebony war-axe. I also use an ebony bow + ebony arrows for sneak attacks sometimes.