I've made reference to this request in John.Moonsugar's MLOX thread, but I've decided it would be best to start its own thread because I don't want to hijack his. It would probably also be best to have its own home in case someone with programming skills decided to run with it. For those who have played either WOW or Torchlight and used their mod download installers, i.e. WOWMATRIX and Torchleech, you'll already know what I'm looking for. For those who haven't...

I've basically been laid up the past week with a torn muscle in my back, so I felt obligated to play Torchlight, which is basically a Diablo type hack and slash. After it crashed on me at the end (right before I beat the badguy) and somehow erased my save game (i.e seriously @&$?ed me off), I discovered a mod downloading and installing tool for it called Torchleech. Torchleech is apparently a clone of WOWMATRIX. As I have never played WOW, I was never exposed to it, so please excuse my ignorance if this request has been made elswhere. Basically what these two programs do is access an online cache(s) of mods from various websites to which they are uploaded, allowing you to install and update them directly from said website. It also identifies conflicts between mods. This is similar to what Morrowing Plugin Manager does with existing mods in a folder on your computer, except in the case of Matrix and Torchleech, the mods are stored online. After installing/unistalling a few dozen mods with simplistic ease, I decided one thing. I WANT THIS FOR MORROWIND!!!!

Think about it. All the mods kept in a dedicated (or multiple) websites that can be easily updated by the authors. You click on Morrowind Matrix and it gives you a list of all the mods available on those websites. You click the ones you want and they install directly on your computer, checking for conflicts as you add the next one you want. And during successive uses of the program, you can check for updates that download and install automatically. If you don't want a mod, you click it and it uninstalls. There's even a place to request mods that users want included in the cache that haven't been added yet.

Now, I don't pretend to know the least bit about programming. But, considering these types of programs already exist, I can't imagine they would be too difficult to reproduce, especially if permission could be gotten to use existing code from them. I believe WOWMATRIX uses mods from a couple of different sites, thus it may be possible to set up a similar connection with existing mods sites like ElricM, TEXNexus, Great House Fliggerty, and maybe even FilePlanet. I'm not going to post any links to either of these programs. Rules, you know. But they are easy to find.

So, the question is, who would be willing to try something like this? I know we have a fair number of programmers around who like a challenge. Of course, I know they are also incredibly busy, too. I wouldn't have a problem volunteering my time to sort through mods if someone could come up with the programming needed. I'm sure that this is something the entire community would assist with implementing if it could be done, as it would benefit us all. Or am I spitting in the wind?

There's a few issues to this, mostly Gollum-modders ("mine, my precious, nasty filthy [players] try to rob us!") who excessively limit their work and just the differences in mod packaging (archive-format, directory structure, etc).
In addition, the amount of disk space and bandwidth required could be prohibitive unless someone has a good server lying around (I'm sitting next to a slower one at the moment, with ~30 gigs free, and (except when the router acts up) it's reliable enough for testing and development).

However, at one point Fligg and I were tossing an idea around for a datafile data repository, allowing players to cross their favorite items and produce a new one (send two ESPs or select from existing mods, pick an item, certain attributes, download a new ESP created by the system with your new item). That hasn't been made yet, but is certainly possible.

The main issue would be organizing the mods and holding them. Leeching from all the various hosts would be very difficult: ElricM uses a php download module, which might be able to be bypassed, but that would take work. TESNexus has restrictions, FilePlanet is a commercial file hosting site, it'll be very difficult to script a downloader. GHF is actually probably the easiest to download from via script.

If someone can figure out how it would work (in the actual organization, not including code), then I might be interested in taking a shot at programming it. Thoughts I have include using SQL (probably MySQL, most servers have it, and I have the MySQL client libraries on my comp) to get lists and such of mods. The only problem is how to add them to the index and where to keep them.

It's probably going to be very difficult to get permissions from all modders to upload their mods to a specific web site - it's been proposed and discussed before with quite a bit of resistance. But this might be something that can be tied into Yacoby's ES Search engine using a front-end that can check for updates to mods against a locally installed db (like mlox) and grab them via ES Search off the existing upload sites.

With WoW currently anyways, several different sites have their own downloaders as far as I am aware, eg curse gaming has its curse client.

To the best of my knowledge it interacts with the directory to find out what you have and matches that against what it has on the server, and then works out if your stuff is out of date or not based on version numbers. I'm pretty certain that it doesn't know anything about conflicting mods, partially because such things pretty much don't exist in wow, at least not in the way they do in Morrowind. Mods might be in conflict with oneanother and mess up your gameplay experience if you don't sort them out, but all WoW mods are User-Interface mods and don't have any kind of evils relating to your 'save game' or whatever exists on Blizzard's server.

Well, I don't know that I'd worry about permissions from the modders themselves. Obviously if they don't want their mods uploaded their wishes would be respected. I think if the work is done to actually create the program it would behoove modders to actually use whatever upload site was created as their primary source of distributing a mod. That way, if they updated it, they could just resubmit it to whoever was in charge, and they would R&R the mod on the server. That prevents numerous requests by mod users as to things like "I can't get such and such to work because..." when they are using an outdated mod. I know a lot of modders prefer to maintain control of where their mods are uploaded. I think this would be the best way for them to do it and not have to worry about redundant older versions out there causing hassles. Modders who didn't want to use the website wouldn't have to, or they could give their permission to upload their mods and someone involved in the project could do it for them. That would save them the hassle and allow the project admins to maintain an updated mod list.
I know that with Torchleech, even if you load mods that aren't on their list, they will show up in the list when you restart it. But, instead of having a short explanation of it next to the mod name it has UNSUPPORTED MOD or somesuch written in. It also still allows you to uninstall that mod. This allows gamers to use mods that aren't on the Torchleech list, but still know what's in the mod folder. So, for Gollum-modders who want to remain on another site or even their own site, no harm-no foul. People can still use their mods, they would just have to load them the old fashioned way. As for leeching the mods, I kind of assumed TESNexus and FilePlanet would probably be problems. When I thought of this, I actually had Fliggerty's site in mind as a host. Its generally well respected and I don't think I've ever heard of anyone having download problems with it like Nexus. As for organizing the mods, its not something that needs to be done all at once. We could start with the most commonly used and add as people request them and modders permissions are received. As I stated, I wouldn't have a problem volunteering time to help sort through them and organize them. I'm just programming illiterate. A lot of them would probably need to be renamed and/or repackaged as well, something else I'd be willing to help with. As for how it would work, I think the simpler the better. Torchleech is pretty straightforward and does most of the work determining conflicts in the background. I'd be willing to try to contact the Torchleech crew to see if their code could be anolyzed and/or used as a base. At the very least, I'd suggest downloading it and taking a look at it.

@tetchy, I completely missed the reference to mlox. It was while reading that thread that I suggested this initially. Didn't even think of Yacoby's search engine. A combination of the two might be a good workaround for this. Or at least a start in the right direction. Some of the conflict detecting coding from MPM might work in there as well.
@peachykeen, I don't think I've ever read anything about the data repository you were talking about. Is that something you've mused about online, or just behind the scenes? Sounds very intriguing.

Yep, I was lurking the mlox thread when you posted there about this. B)

Not sure if you remember the http://www.gamesas.com/bgsforums/index.php?showtopic=977412 from early 2009 (you posted to it a couple times) - that would be an example of some of the modder resistance I mentioned in utilizing an unestablished web site for a centralized repository (mainly because the author was new to the community and new web sites need time to become trusted).

I still think it would behoove anyone trying to develop this to figure a way to grab mods from the existing sites; no need to relocate/re-upload a mod, it avoids having to seek additional modder permissions, and gives modders the freedom to choose which site hosts their work.

Yeah, but I wouldn't really call GHF an unestablished site, if they would agree to hosting something like this. And for those modders resistant to the idea, well, I think if this were to work anything like Torchleech or WOWMatrix does, it would be their loss. If a way could be found to leech from those sites, I'm sure that would be much easier to do then re-upload all the mods to another site. But that doesn't solve the numerous types of compression files, weirdly named mods (I have a couple with a mile of numbers before and after the title), or sorting and organizing them all. Although I guess it would be rather simple to alphabetize them. And that doesn't even begin to address huge mods or those with .bsa files, which I seem to remember had a few extra loops to jump through to install them. But this is just an attempt to get the idea out there again, especially now that we can point to similar programs for other games that seem to work rather well. And if someone like peachykeen is willing to attempt coding something like this, I think we'd be surprised how quickly people would jump on board.

I'm aware of the huge number of mods out there and how much space they take up. I have 3 dual layer dvd's filled with them, and I don't think that even scratches the surface. Which is why it would be nice to see them organized somehow, somewhere. I'm not talking all of them. Maybe just 500-1000 of the most commonly used. You know as well as I do there are some real gems out there, as well as some real turds. (Yes, I'm still creased about the one that permanently trapped me in a building because the supposed uberweapon I was seeking was a trap because the author didn't like cheating, or some BS.) I remember a thread from a while back that had people posting their mod lists and someone in it actually went through to figure out how many people were using which mods. I'm willing to bet 1000 would cover most of them. Hopefully this will go further than the Morrowind Mod Manager thread.

My disk space and bandwidth at GHF while plentiful for what we do now, would not be sufficient for such a project of this scope.

That being said, I've been working on turning my shed into a bit of a datacenter. I'm currently in the process of burying a conduit to run some cat6 cables out there to connect my servers. I've got a couple of decent Xeon dual 2.4ghz servers with lots of disk space in them. And as soon as my ISP gets the new fiber line put in that they've been promising me for months now I'll be set to handle whatever is needed. Just let me know who will need access to the disk space and what resources are needed and they will be set aside.

I have a database of some of TESNexus and Planet Elderscrolls. About 22,000 mods and 23,000 download locations (and still growing ), I also have mod version information for some sites if anyone would find it useful. It is usually slightly out of date though (~15 days for mod updates, although new mods tend to be added to the db faster). I am happy to either make database dumps available or put together a SOAP or JSON based RPC server. Or both.

It might allow you to be able to access site data without having to put as much load on sites. The one thing is I don't think Dark0ne would be happy if you put together a automated downloader for his site. He exists on ad income and bots don't click ads (And lets face it, bandwidth isn't free and the last thing anyone wants is a great site like TESNexus to go under). He may be willing to offer a service to premium members :shrug:. I am not sure about Apy and PES but you may find the same thing, as he requested I didn't link directly to the download uri with my search engine (not that I was planning to).


I have some (probably bad) C++ code from ages back that is fairly good at sorting mods. I don't think it was particularly hard.

I personally think this is a terrible Idea & would never support it .... usually anything that has Leeching involved is not good for the "target" in the long run.

All this will do, if successful, is hurt or kill hosting sites like PES or TESNexus or even Fliggerty's site if he isn't the host of this thing. ... & thats just what we need something that kills hosting sites when they are already thinning out.

Yacoby's search engine gets you all the info on mods & their various DL locations .... so what if you have to install them yourself & maybe have enough intelligence to read the readme & follow the usually simple instructions.

I'm sure torchleech is great for Torchlight ... but Torchlight will be forgotten in a year or 2 max ... Morrowind is nearly a decade old & still going strong but something like this could kill it ... all in the name if a little convenience for lazy people ... IMHO

Hmmm, so I'm going to guess Darknut is one of those pockets of resistance previously mentioned. Its hardly my desired intent to drive sites like his down the tubes, as they are basically the foundation of Morrowind modding. So from what I gather, an automated downloader that leeches (I really hate that word) a mod from a site skips over the ad scripts that allow some sites to generate revenue. There's no way around that? Kind of an automated clicker that says the site has been visited? As I've said, I have no clue about coding, so I'm not sure what is possible, or easy/hard. It would be nice if there were a way to "give credit" to the sites that actually host mods that the author would rather not upload to a centralized database. That way they could continue to host mods in the future, and they might be willing to participate in the download installer project.
Some of the reading I've done on WOWMATRIX comes down to basically the same thing. Money. Sites like Curse are cutting them out because they are free, while Curse is trying to charge for a service. Don't know how that will work out, since WOW basically has a similar "no pay for mods" rule like Morrowind. As for Torchlight being forgotten a year or so down the line... maybe. I hear they are working on an MMORPG. But WOW has been around for awhile, and I don't see them going away.
There has to be a happy medium somewhere, without calling people lazy to get your point across. There's nothing wrong with trying to simplify a process that confuses a lot of people, especially when, as Yacoby notes, he has a list of 22,000 mods out there. What's convenient about having to sort through that?

@Fliggerty, thanks for the offer of server usage, once you get up and running. With that, plus the possibility of peachykeen coding a program, this download installer could possible get off the ground, even if not envisioned exactly as I originally conceived.

So how about this for compromise? We incorporate Yacoby's search engine to locate mods, as he has already done a huge amount of work and seems more than willing to share it. Anyone who wants to put their mods on a Fliggerty based central database can do so and use the auto download installer to install the mods directly from the server. Maybe throw some coding in from MPM for conflict detection. Those mods not on Fliggerty's central database, i.e. like those on Darknut's site, TESNexus, etc, can be linked to but the download and installation would have to be done the old fashioned way. Torchleech has a button or two that allows you to open a mods webpage. Something similar could be done here. On the program these mods would have "Unsupported Mod" in the description column. The program would still recognize them, just not support the auto-updates, etc. if a newer version is released. That would have to be done manually. Again, this would not have to be mandatory participation. And its still all theoretical. But I think there are enough advlts here that a beneficial compromise could be found.

Part of the fun and thrill, for me, is searching for just the right mod for me. ES Search has made this process soooo easy, I can't see why mod downloading needs to be simplified even further. Besides, the way mods are packaged (folder structure) isn't standardized; neither are player's operating systems (I hear Vista likes to hide things), nor local disk drive (usually, C:, might be something different). Besides, I feel so smart knowing how to install mods! :nerd: With that new Mash thing (I forget what it is called!) you don't even have to take mods out of the archive to play them!

I, for one, would never be comfortable with just having an Internet program automatically sniff around my PC to find where to install stuff. It'd be easy to create a virus or other malware to get inserted. Maybe I'm being paranoid, maybe not.

To be honest, I haven't really read all of Shinjack's posts because they are ginormous walls of text.

The ES Search is a exellent web engine to look after the mods you want. In your opinion of Torchlight, I agreed with you.

The problem with that would be downloading mods that aren't in the system.
However, it might be possible to make a program so that index could be browsed from the player's computer, without using the search engine. How big is the db and what system are you using for it? Any chance I could have a copy?


Running it on a private server would definitely be better. Seeing as Fligg will have functional servers and I have one suitable for testing, that shouldn't be an issue. Except for the recurrent problem of selfish modders. :shrugs:



Mind sending it over? Did it support archives (and if so, what dependencies does it have)?




If you didn't notice, it's generally been suggested the program not leech, but instead use a site dedicated to this purpose.

As far as killing Morrowind with a mod downloader... That's just ridiculous. Personally, I'd love a tool with a metapackage so I could download and install Better Bodies and such after reinstalling with one click, instead of going through the process I've done 2 dozen times already.
People who know how to install mods won't be affected by this (except maybe saving time), people who don't will be more likely to use mods and because of that, more likely to stick around. Chances are, it'll only draw more people to Morrowind, if the technical mess is removed.

Yeah, I tend to over elaborate. My bad. To be honest, I never even considered different operating systems (mainly because I wouldn't touch Vista with a ten foot pole so I'm still on XP), so you have a valid point there. Along with the potential for viruses. Sometimes I forget that there are less than scrupulous people out there. I've never been able to get WyreMash to work properly on my computer, so I can't comment on it other than the positive reviews I've read.

@peachykeen, I have to agree about drawing more people into Morrowind if the technical mess is removed. Last time I started a game, it took me three weeks to sort through and load/remove conflicts from mods I wanted to play. Just a little bit of a hassle. Probably my fault, though, for having too many mods on disc. But, I... can't... help... myself...

Just thought I'd add this pic of the Torchleech interface:

http://i4.photobucket.com/albums/y131/Skinjack/Torchleech.jpg

It gives a general idea of its layout and functions. The first thing it does when it loads is checks the mods database to see what you have loaded and what is available, then generates a list. You can see different info about the mods, i.e. author, rating, category, even a brief description. Down below is the host website. There are a number of other buttons as well, to request a mod be added to the server list, auto updating, conflict detection, etc.

OSes wouldn't be too huge of an issue. A program could check for Morrowind's location and warn the user if it's in Program Files on Vista/7 and tell them if it can't isntall things to there.


For the most part, Morrowind mods are pretty good. I don't think I've ever run into one with a virus. IF they were held on a server meant for this, it wouldn't be hard at all to run a virus scan on them before putting them up there for download.


I know perfectly well how to install mods, and when I'm testing things and have to reinstall, it can take months before I bother to put a head replacer in, just because I don't want to go through that same thing over again. It gets boring, copying this and that, enabling, cleaning, testing...
Plus, some of the people who haven't tried mods because the whole install process intimidates them will learn what they're missing out on. It'll invite the less technical (and possibly more creative) into the mix.

It really depends on how much work it'd take to put it together (no point if it'll take more work to build it than just installing the mods would've).

Overall, I think it'll be a good thing. So, I'm tempted to make it for myself, because I want it to simplify things, and a pox on all the selfish, stuck-up modders who won't learn to share.

I prefer tight security if my PC is connected to the internet. I've the same attitude as you Alaisiagae. :foodndrink:

I must ask peachykeen & Skinjack one question; Are you willing to pay these servers from your own pockets ?

I'm sitting next to a server I own at the moment. There are no additional costs, since the server is essentially free and the bandwidth it would run on is part of the monthly internet plan I'm using.

I'm sure your right in MW would not totally die .. but if sites like PES & Nexus go away it will dwindle significantly .....

The bottom line is none of the Big hosting sites will ever go for something that directly leeches files from them without the user having to login .. why would they .. they would be stupid to do it.

I still think having a bunch of automated BS will just invite problems & could turn MW into a security issue ...

Obviously what I think doesn't mean squat in the end ...but for the record ... I think its a bad plan .. & will do way more harm than good ... IMHO

I've come to the conclusion already that certain people will never be happy with this concept, for whatever reason. C.Carjades seems to be one. I can understand Alaisiagae's hesitancy and points of view. At least he gets his point across without sounding like he has an ulterior motive. Darknut might be willing to come around if, instead of leeching from his site, a direct link was provided through Yacoby's ES Search incorporated into the program so he would still get his ad support. I'm afraid I just don't understand the resistance to something like this. It would be like any mod. If you don't like it, don't use it.

Operating systems are not a problem. Most carefully written programs it shouldn't matter what the drive is called to the user (Allocating letters is stupid anyway, unix did it better)


I believe MWSE, FPSO etc does exactly the same thing. For the most part checking where Morrowind is installed is as little as checking some registry settings and if not prompting the user to select the location. I am sure most of the programmers here would be happy to open source it if it gets written. That way anyone who wants can check the code and compile it themselves.


MySQL, it is about 30mb uncompressed. I assume most of that is descriptions and me not having a fully normalized database. The http://search2.yacoby.net/db/


Sure. I think it supports everything (almost everything). I think I bundled universal extractor with it.


Or even better just check if the directory is writeable and if it isn't ask the user to run the program as a root user (admin? IDK what windows calls it). That way it is a bit less windows centric.

This is true ...

That's it, I'm out of here. Bye !

I completely agree there. Leeching from those sites would hurt them and be a pain for the programmer. It's far better to either have a private server (when the file could be hosted by us), or provide a link to PES/Nexus when it couldn't. That take care of your complaint there?


All measures will be taken to prevent security issues. No private data would be uploaded, nothing like that. What mods the user downloaded would be their business (although an advlt category might be in order as a warning), at the most it would increment a counter on the server (just a number of total downloads or per-week).
Very little would be automated, and most of that is stuff that's been automated since the dawn of time.
Know how Tribunal and Bloodmoon know where to install (and Better Bodies and a dozen other mods with installers)? It's a simple registry check, completely safe, and would be done the same way here.
Obviously if a private database were kept, no mods would be allowed up without being thoroughly virus scanned (I have access to 3 different scanners at the moment, Comodo, AVG, and Norton). There are plenty of freeware virus and spyware scanners, and it might be possible to have any mods scanned when they're downloaded, before installation (I'm not sure exactly, but I think many anti-virus programs have a special module any program can call to scan particular files).
Obviously, a little bit of internet sense would be required (don't click everything, especially if it looks funny), but the Morrowind community generally doesn't seem to be busy throwing viruses at each other, and care would be taken to keep this system as clean as possible.


Some people don't like certain ideas. However, a few good points have been brought up. Hopefully we can address those and figure out a good solution that makes the majority happy and helps as many people as possible. The minority who still hates the thought, well... I don't mod for anyone else, I just sometimes share what I make with the community. This would be like any mod.



Funny thing is, Windows doesn't use letters itself. I think that's mostly a user-friendly feature (which, I have to admit, it's easier to say the Recycle Bin than C:\$Recyc\[date]\[index] or whatever, where it really is, or \dev\null). Win does refer to the drives by indexes (HD0\Part0, IIRC). The SDK has more info, in the mounting drives with handles page.


If I were to do the coding for this, it would most certainly be open-source. Not only does that allow anyone to help further it or fix it (or check it themselves), it also makes it a contribution to the community, to learn from.



Will download the db. Got universal extractor already, and got your PM. Will reply in a moment.



Windows has admins and a root user ("Administrator", only visible in safe mode). Something like that would work fine, after the location was grabbed from the registry.