I know it is simple enough to have specific explore music in a cell. Just have only silent music files in the explore folder and have the music scripted to play in specific cells.
But is it possible to have that music keep playing when a combat starts, so the music will be continuous?

Not using scripts. You may want to look into FPS Optimizer. If I remember correctly, it has several options for music, possibly including an option to not interrupt music during combat.

Dunno if this'll help, but I've been using this for awhile: http://planetelderscrolls.gamespy.com/View.php?view=Mods.Detail&id=7899.