I'm not that far in only been to two towns but I see alot of threads about smithing and I was wonddering where you find the ore??

In rocks. Some rocks have colored streaks, and can me mined if you are carrying a pickaxe.

I ran across some in the first dungeon "Black Talon I think?" and it was just luck I found it. I couldn't tell the difference between the ore and anything else around it!

I know it's more "real" to have it like how it is, but damn it's hard to see!!

Check at the very tip of the throat of the world... You'll also find a unique pickaxe there