I stole the elder scroll and got out of the palace, but now I don't know where the Gray Fox is at! Where is he?

In Anvil IIRC.

No, I found him. He was in Orthelo's house in the Elven Gardens District. But it's stupid because I took a different route to exit the Imperial Palace (i.e. not following the red arrows on the map) and I managed to escape but I guess it didn't activate the grey fox to appear in the house. Kinda dumb how you have to do things a certain way in order to activate the next command. I guess that's the trouble with computers, eh?
I found him, though.

In my game I'm using Frostcrag Reborn which comes with a spell that transports you to Frostcrag. I wanted to be able to use Jaks boots in the rest of my game so when i got the scroll I used the spell. Well the game didn't like that and it wanted me to still go down the fireplace. I figured out the trick for me was to cast the spell just before I hit the bottom and it would update the game saying i made it to the bottom.