yah, scale your difficulty up.
You can't complain about the game being to easy when you haven't even tried the harder difficulties. Some games they don't make much difference, or just turn AI into super aware super accurate headshot bots.... but that's not the case here.
Master difficulty WILL kick your bum.... You should try Expert though... just a step up and should make a significant change to dragon/boss fights.
Personally I'm lovin skyrim. Fantastic RPG. Not unnecessarily complicated, great atmosphere, stunning graphics (considering the hardware), and both allows you to totally explore the vast world on your own or lets you follow the cookie trail main story quest if you want a shorter game.
Skyrim is what you make it.
So if you want a bigger challenge... make it more challenging