You can find dragon armor, but it will take you a long time to get it. Smithing it is definitely much faster. Same with daedric.
This. The gear that appears as loot on the enemies you kill levels up as you level. IE: A bandit "chief" you kill at level 1 will have fur armor, maybe a piece or two of hide/leather. That same bandit chief, killed at level 20 may have Elven armor instead. Thus, once your character achieves a high enough level, you will start finding Daedric/Dragon armor on your enemies. Oblivion had the same system. However, in Oblivion, you tended to get to a certain point where EVERY enemy became a Glass/Ebony/Daedric loot pinata, and you could gather several full sets of top-tier armor just by running one dungeon... They hopefully learned from this, and made Daedric and Dragon armor much harder to obtain in Skyrim. Since you have the alternate option of leveling Smithing and obtaining the same gear quickly but at a cost in materials, there are two balanced choices:
Smithing: You get your daedric/dragon armor quickly, and possibly even earlier than you really need it, but have to invest time to level, and buy a ton of materials.
Looting: Your daedric/dragon armor is free, but spawns slowly, randomly, and only at higher levels.