Yes, great idea. I love scholar characters, probably because that's the closest I would ever come to an adventurer: a highbrow, don't-get-my-hands-dirty, academic type.
Some things about playing a scholar in Skyrim can be difficult. But here are some ideas I've tried or thought of in the past
-Scrolls are a perfect complement to the scholar, and some are quite powerful (but may need to be used with care around companions). Definitely make them a priority in loot and when in the marketplace.
-Staves *may* fit, depending on the character. If you don't see him as a real wizard type, you can always collect them, use them in a pinch, and sell them once you've studied them enough.
-Speech is a nice skill for this, since you'll be buying soul gems (filled and unfilled) and possibly even buying enchanted gear to learn a rare enchantment - and buying scrolls, too.
-Lockpick might be worth a mention. You could invest 5 perks and get Treasure Hunter, which comes with an increased chance of finding weapons and armor (enchanted or unenchanted) in most chests.
-Given those things, I don't think you'll have too much trouble with money. And if you find yourself poor, well, enjoy it! Skyrim is throwing gold at you most of the time, so revel in being a poor scholar.
Dealing with enemies - Skyrim is combat-heavy, so there's always the potential for a fight (or ten). You can roll with this, relying on your hireling(s) and crossbow (plus scrolls). Or you can invest in some crowd control like Illusion or Restoration (turn undead). Personally, these seem a bit too flashy for the kind of scholar I think you're going for, but I'm just throwing it out there.
General roleplay stuff -
-Spend some time in towns talking to people and getting the lay of the land. Did you glimpse a ruin on your way into town? Does anyone in that low-class tavern have any information on the surrounding area? Does the jarl or a court wizard have any knowledge to share?
-Stay a few days, keeping a room at the inn and going out for expeditions each day.
-Books. Does the storekeeper have any dusty old tomes that might point to adventure? Or books of general historical interest? Spend money and haul them around to read.
-The College has some great quests for you! I'd do Saarthal and perhaps Hitting the Books, but then stop the main quest altogether. The first two are about archaeology and book retrieval; after you go to Mzulft, the questline ramps up in a major way, so I just never go there. But even if you don't touch the main College questline, you can get a lot of mileage out of Urag's side quests, since they're all about finding books! Shalidor's Insights and the other radiant one where you have to find some random book - they're pretty much ideal for a scholar.
-Learn the dragon language! I did this with my Imperial Battlemage. Sketched out the symbols on every word wall I came across, and cross-referenced the familiar symbols with what I had already seen. It was a really nerdy and fun mini-game I made for myself.
Skills
Sounds like you have a lot of this figured out already. I'd go with Archery & Enchanting (obviously), Destruction (grab Dualcasting and Rune Master), Speech (possibly everything here), Lockpicking, and maybe Alchemy as a minor skill, as you mentioned. With Enchanting, you can carry around an imbued dagger or two for an emergency, and you won't need to put points into One Handed, so I'd stay away from that skill.
I think you'll be surprisingly effective! Good luck