Don't you think it would be nice to not have to look around everywhere for that one little ingredient you need? Of course you wouldn't have to plant everything you come by... you could just plant those rare and expensive ingredients. Also, your home cultivated ingredients could have an effect bonus applied, or replenish faster than wild ones... Or it could be the other way around, and certain ingredients could be more potent if harvested from the wild. For example: "Cultivated Mandrake Root" would be more potent than "Mandrake Root", but "Cultivated Flax" would be weaker than "Flax".
Has a gardening mod in Oblivion who works pretty well, as other say some plants are hard to find some places, suspect that vegetation will be even more localised in Skyrim. Serious doubt they will add it but it was moded into both Morrowind and Oblivion.
Only used the Oblivion version who was pretty sweet, activating the gardening tool placed a sack in front of you, place plant ingredients and it turned into a heap, over the next days the plant grew and could be harvested, at this time it was a standard Oblivion plant like any other.
Downside was that it was dependent on cell respawn to regrow so you could not have a farm you lived on; you has to place the garden spot a place you visited every 3rd day to harvest and perhaps plant more.