Just a thought on a role play level of things when designing a character.

I was thinking that mages have spells to fortify themselves, muffle their sound or become invisible etc and many potions are for putting on a blade which a pure mage would not use. Rogue in the pure form don't have anything to fortify themselves and they need low armor or clothes to assist their sneak because they need to be quiet and move easy. Making potions to poison a blade, heal oneself or create an illusion to me seems better for role play suited to a rogue or a ranger than a powerful wizard who performs great spells.

Rogues also in theory shouldn't have the strength and stamina of a warrior even though they still use physical weapons so ideally could use a good potion to cheat all the hard work needed to enhance the body's strength that a warrior would need.
Alchemy is mostly about collecting herbs which again to me seems more logical to a rogue such as a ranger style of role play. If anything it is also a form of drug trade (the legal type and not just the illegal type) so again suits a criminal guild such as the Thieves Guild. Assassins use potions for illusion so again an example of rogue role playing. Rogue classes in many ways are wealth orientated whether it be a thief, assassin or general charmer looking to manipulate a buck so would sell their wears to a merchant to make money. Mages however would enchant items and make and sell scrolls. warriors would smith an item.

Mages obviously find potions as an great asset the same way a warrior does but I just wondering if making drugs and poisons was better suited to a rogue style of role play than a mage?

I think it could go either way. Skyrim is the first game to make it feel the way you described, a thing for scoundrels and assassins. In past games it had been much more of an arcane art it seems though.

Poisons make up half of alchemy, and serve no purpose to a mage. Potions are very important for all classes since they keep you alive.

I would agree that Alchemy is better suited to a "not mage" class since they would benefit from both the potions and the poisons.

All the crafting skills really could be applied in one way or another to the "Basic 3".

Enchanting - Anyone could use an extra damage or protection
Smithing - Anyone could benefit from stronger weapons, armor, plus jewelry
Alchemy - Anyone can benefit from any type of potion that replenishes health/stamina/magika or increases another skill.

I would think Smithing is definitely the most class specific because while you can smith light armor for mages and rogues, there is no benefit to a higher smithing skill for jewelry.

Enchanting is definitely better suited to mages because of the magika enchantments as well as the school-based cost reductions.

Alchemy is definitely better for thieves because of the poisons and how well the couple with stealth.