So I've noticed that in the Temples of Arkay the entry way always contains a pile of loot each time you enter, generally having gold and some item - alchemic ingredient mostly.

I've been picking this stuff up as I've been doing missions, but I'm curious if I'm technically stealing Temple property?

That's perfectly safe. You'll get the "Are you sure you want to look through these personal belongings" message if you actually attempt theft.