I took the advice from people here to level pickpocketing as a means to level my smithing up. That is, pickpocketing the gold back from Gray-mane after having trained.
I have 71 in pickpocketing. +87% Pick pocketing from enchanted gear pieces, 20% bonus from the first pickpocket perk AND I have the 50% easier to pickpocket gold perk (Cutpurse)

I still have ZERO percent chance to steal the gold back even from just one skilltrain (Which is about 1700 gold atm). Yet people insist this works. Is gray-mane bugged? Should I try another smithing trainer? Should I try to pickpocket him while he sleeps?

Im at a loss atm on what to do, just spent almost 2 hours to make this work and still that cursed 0% chance to steal haunts me.

Sorry judas, the "gots" are on his side.

According to UESP :

Trainer pickpocketing: Pay a Trainer to train a skill, steal your gold back, pay again, and repeat for as long as you're able to "buy" new levels of training. This way, you can gain as many levels of the skill as you wish to learn from the trainer, and gain levels of Pickpocket with a net cost of 0 gold. With a high Pickpocket skill, you can train up to level 51 without Pickpocket perks, or to 76 with all of the perks. After this, the gold cost becomes too expensive to Pickpocket back.

1700 is too high to pickpocket without getting lots of enchanted items and perks.

Trainer pickpocketing exploit is most useful for getting to level 51 when the training costs are under 600 gold. After 51, the training gold jumps up to 1500.

After 51, you can sell him stuff to get your money back. I think it's called the trainer-merchant exploit. Enchanted daggers will do nicely. If you have the merchant perk, you can sell him anything, including expensive potions you made.

[edit]

More info:

The formula for success is spelled out in the UESP page: http://www.uesp.net/wiki/Skyrim:Pickpocket

According to the formula, if your calculated chance of success is less than zero before enchantment or potion, no amount of enchantment or potion is going to help you.

At the moment, your chance of success is -14% (40% + 71% + 25% - 170% + 20%)

If you take one more perk in Light Fingers, it's going to raise your chance to +6% and it's going to be boosted by your enchantments and cutpurse perk, so it's going to be 16% (6% * 150% * 187%)