I haven't bought a horse but the other day I was treking to the Temple of Azura from town and noticed a horse trotting along the path. I clicked on it and it said "Ride Horse" normally it shows as being in red as steal horse. I've noticed a few more random horses out on paths in the wild. Are these wild horses that you can just take? Or are these horses belonging to dead NPC's? I rememberin Oblivion if an NPC died and its horse was left alive it was still classed as stealing to take the horse.

I've found a few as well. I think they used to belong to dead npcs. If they did, I'm glad they changed the "stealing" thing. It was annoying in Oblivion to find a dead soldier with the horse just standing there and not be able to use it because you didn't want to steal it.

You can take them but you don't own them, and as soon as you dismount they wander off like a stolen horse would. Oblivion had wild horses, I think they acted the same way (wander off if you dismounted).

Be nice if a mod let you "tame" a stray or stolen horse by keeping it fed with carrots every time you dismounted to get it to stay if you ever ran out of carrots to give it though, it might decide to no longer stay with you...

Not too sure, but i think it depends on what side you play.. Stormcloaks/Imperials. I chose to like Stormcloaks and just about every stormcloak horse is available to use without stealing, once you dismount, the horse returns to the owner or place it was found standing. (i could be wrong as always)

I met an Imperial on the road east of Riften, she attacked me without reason, so obviously I killed her. 5 seconds later I noticed a stray horse that I could ride without stealing. I think it belonged to her.

Cool its goign to be quite handy to just use a horse for a one off journey every now and again without having to buy a horse. Mind you the horse I saw ended up being killed further along the path because it decided to take on 7 mages and an orc..lol

Meh, I had me attacked by a bandit but his horse was on steal afterwards.
Edit: encountered numerous free-roaming horses later on and they were all on steal. This game leaves way too much on steal even when there's no apparent reason for that. Annoying.
While at it, I just turned a valve in dwemer museum and the entire Marcarth police department doesn't even ask me about my bounty - they just charge. Idiots.