Almost always, this is a sorting issue. An ESP/M that relies on another ESP/M is out of order, and it marks all subsequent mods that rely on it as out of order as well.

There should be two files associated with GIANTS Ultimate v2.7.1 (I assume that's the version you're using): the GIANTS ultimate ESM, the GIANTS ultimate control file ESP, and now you are adding Z4K's add-to-level patch, right? I don't know the original file names, as I accidently synced them to my project after changing the name in /Data Files (so much for the official-local.csv file!).

Make sure the dates put the load order like this...

• Morrowind.esm
  
• Tribunal.esm
  
• Bloodmoon.esm
  
• GIANTS Ultimate.esm
  
• Giants Ultimate Control File.esp
  
• Z4K add-to-level patch.esp

Do you use MLOX to sort your load order? If not, you should - it will sort the mods in the best order; although, even it's always a work in progress.

-RMWChaos

Ohhh ... it all makes sense now.

Turns out the problem was with Tejón's Fatigue Effects. Mash just didn't like that little ó, which is NOT ascii standard, in all his mod names.

Okay, and you said it's fixed in your current code; so all is right again with the world. :celebration:

Danke,

-RMWChaos

Wow - ok third time - load order is fine - yes mlox - just like BOSS with Oblivion. On it from the get go.

I'm new to morrowind - old to Oblivion/Fallout3.

Yes that load order and yes checkboxes still orange - when used with the cleaned Tribunal/Bloodmoon using your tes3cmd bat file (the last one). Not orange with the original uncleaned Tribunal/Bloodmoon.esm

That is the issue - it is strange - hence my posting about it.

LOL, sorry man, trying to help out here. I think I understand your problem, now. Try this ... if you have already, please forgive the redundancy ... not always easy to follow what's going on remotely.

• Left-click on GIANTS.esm
  
• On the right-hand side is the list of masters it requires. Check if any of those listed have a Yellow (or Orange) box - probably both Tribunal.esm and Bloodmoon.esm do.
  
• Right-click on Tribunal.esm (if you get a message "Edit the masters...?" just mark the checkbox not to notify you again, and click ok. You'll have to right-click on Tribunal.esm again.
  
• Left-click "Change to..."
  
• Find Tribunal.esm in your mod list and left-click on it.
  
• Click "Open".
  
• Do the same thing for Bloodmoon.esm.
  
• Now left-click "Save" at the bottom-right-hand side.
  
• Do the same for any other mods that show Yellow or Orange boxes in your mod list.

That should clear it up.

I think tes3cmd has a way to modify the master so that dependent mods do not notice it's been changed at all. Mash does it by updating the mod masters list instead. Let me look into it, and I'll update the tes3cmd.bat file.

Let me know if that works, though, to confirm it is the problem you're having. Then I'll know that the change to the tes3cmd.bat file will correct it for you as well without having to update the masters for every single mod you use that is dependent upon Tribunal and Bloodmoon.

-RMWChaos

Yeah that works as I reported above - what I was not sure of was whether this would be masking another problem.

After cleaning Tribunal/Bloodmoon esm most mods show those masters as yellow and therefore were themselves yellow.

Updating masters fixed that then I see the orange - and not sure what that meant as the readme does not say.

I think what is the cause was that I originally cleaned GIANTS (and related patches) with the uncleaned Tribunal/Bloodmoon esm then when I did clean the Tribunal/Bloodmoon esm those three then caused Tribunal/Bloodmoon esm to be orange in the masters.

So I'm happy with the short answer that it is OK to just update as with yellow, but I'm still curious what the orange means.

off to work - be back later.

Ah, ok. Yeah, you probably want to redo that, and clean GIANTS.esm after you clean Tribunal.esm or Bloodmoon.esm.

While updating the masters will clear the Yellow or Orange flag, and while it probably won't really hurt anything in this case, it's still a best practice to clean the Official Masters first, then any subsequent mod masters that rely on those, and then any plugins that rely on those mod masters. Er... official masters--> mod masters--> plugins ... yeah, that's right. :wink_smile:

Now go earn that money!

Cheers!

-RMWChaos

DOH! :facepalm:

Okay, so you're correct that John's code does have some checks in it to prevent cleaning of certain files...

# plugins which are never cleaned# name -> reason for not cleaningmy %CLEAN_PLUGIN =     (      'bloodmoon.esm'             => 'Bethesda Master',      'fogpatch.esp'              => 'does not need cleaning',      'gmst fix.esp'              => 'intentionally contain Evil GMSTs',      'gmst vaccine.esp'          => 'intentionally contain Evil GMSTs',      'mashed lists.esp'          => 'does not need cleaning',      'merged_dialogs.esp'        => 'does not need cleaning, but you should delete it anyway',      'merged_leveled_lists.esp'  => 'does not need cleaning, but you should use "tes3cmd multipatch" instead',      'merged_objects.esp'        => 'does not need cleaning',      'morrowind.esm'             => 'Bethesda Master',      'multipatch.esp'            => 'does not need cleaning',      'tribunal.esm'              => 'Bethesda Master',     ); 

Yet, as I previously mentioned, tes3cmd definitely cleans both tribunal.esm and bloodmoon.esm ...

And I've been banging my head against the wall trying to figure out why the --backup-dir option wasn't working...

The optional backup directory...@MODOPT = ("backup-dir", "hide-backups")...and the default backup directory...$opt_backup_dir = "$TES3CMD_DIR/backups"

So why wasn't it working?

Well, it really must have been way too long since I've played around with DOS commands, because I should have remembered this.

When you run a command like this...

tes3cmd clean --replace morrowind.esm

...at the command line, then no problem, it spits it out the response...

Skipping morrowind.esm: Bethesda Master.

However, if you run it from a batch file, and are using %1 to specify the input instead of the plugin name, then it passes incorrect values to the command executable.

There is a way around this, I just have to think about it a bit.

The way you're calling tes3cmd from Mash and passing [selected files] probably won't run into this same problem.

Off to dinner, then will play around with this some more.

Regards,

-RMWChaos

Python 2.7, Latest version of the Unicode wxPython release for Python 2.7. If you want somethings to work you also need to install comtypes as it is no longer included in python 2.7


Yeah. I possibly could automatically save the log if you thought it would be good?


Ah. Point. This really an issue with tes3cmd. The 'evil' GMSTS that exist in the expansions shouldn't be treated as evil by tes3cmd. They can then just be pulled out of future mods as they are duplicates. I should point this out to John (I am not sure if he around any more though)


I think it is just a simple name based backup. But then IDK. I don't know about that area of code.


Point.


Regarding the problems with batch files. I don't have a clue (I don't use Windows and have never done anything in any shell scripting languages) so I am not much help there.

Okay, Psymon, I updated the http://www.gamesas.com/index.php?/topic/1184424-q-cleaning-mods-with-tespcd/page__pid__17647367__st__76#entry17647367, which should take care of the problems you were having, as long as you follow the order of cleaning laid out in the batch file comments. That should tide you over until Yakoby decides whether to enhance the functionality of tes3cmd integrated into WMSA.

As for Yellow vs. Orange, I believe it has to do with a cascading effect of unsynced masters. So if you have a plugin that relies on a master that is out of sync with its master, then you will see Orange, not Yellow. But if you have just a single plugin with one or more masters out of sync, it will be Yellow. Then again, it might be a double-sync problem ... not only is the plugin out of sync with its own masters, but the masters it relies on are out of sync with theirs as well. Either way, it's all about syncing things up, my man.

Well, that explanation sounds good, anyway, but would have to test it to find out if it's acurate or not. Damn, I can spin. :toughninja:

That's all I plan to do with the batch file for now. Going to focus on dumping this oil tanker's worth of Wrye's and Yakoby's code into my shotglass of a brain for a bit.

Cheers!

-RMWChaos

Thanks I took discussion over to that thread kind of off topic here.

Yacoby-

With Wrye Bash if one were to have it open then in windows explorer move an esp out of the data folder that was active - one would only get a message saying it was missing as is what happens with opening this after an active esp has been removed since the last time it was open.

What happens now is if an active esp is removed from the data folder while this is open - as soon as you click on mash to make it update (be active window) this error occurs:

Traceback (most recent call last):  File "F:\Bethesda Games\Morrowind\Mopy\masher.py", line 2428, in RefreshData    if mosh.mwIniFile.refresh() | mosh.modInfos.refresh():  File "F:\Bethesda Games\Morrowind\Mopy\mosh.py", line 3022, in refresh    self.unload(loadFile)  File "F:\Bethesda Games\Morrowind\Mopy\mosh.py", line 3134, in unload    mwIniFile.unload(fileName,False)  File "F:\Bethesda Games\Morrowind\Mopy\mosh.py", line 2394, in unload    self.refreshDoubleTime()  File "F:\Bethesda Games\Morrowind\Mopy\mosh.py", line 2334, in refreshDoubleTime    mtime = modInfos[loadFile].mtime  File "F:\Bethesda Games\Morrowind\Mopy\mosh.py", line 2718, in __getitem__    return self.data[key]KeyError: 'Quieter_UI_Sounds.esp'

IN this case it was Quieter UI Sounds, but I see this error a lot.

When it occurs mash must be closed and reopened for it to update the missing active esp.

.....

I'm new here and not trying to be a major nuisance, but man I want to just start listing feature requests. I understand you are one guy, but so much has been done with Wrye Bash that it is hard to keep quite with requests here.

Things like adding the ability to list mods (like bash does), Add bumpers to installers tab, and an INI tweaks tab. But all that is minor to the one feature that I so miss and that is drag and drop sorting of the mods and BAIN tab. So if you deem it a worthy request - I know many would be much appreciative.

Thanks for considering and thanks for the work you are doing.

Psymon,

Usually, Mash updates the Mods tab automatically any time you add or remove a plugin from the /Data Files folder manually.

I just tested with WMSA 85 open on Mods Tab, opened /Data Files folder in Win Explorer, cut bcsounds.esp and pasted it in another folder, and it was automatically removed from the Mods tab without any message at all.

Next I tested cutting/pasting bcsounds.esp back in to /Data Files with Mash Mods Tab open, same thing, now it added it back into the list without any error or message.

Just to be thorough, I tested again, and this time had bcsounds.esp highlighted in the Mods Tab first before removing it. No errors.

I'd say your WMSA installation is either borked, or perhaps you might try installing it a little differently...

I've tried installing WMSA in three different ways...

• /Bethesda Softworks /Tools /Wrye Mash
  
• /Bethesda Softworks /Morrowind /Wrye Mash
  
• /Bethesda Softworks /Morrowind (originally intended install location by author Wrye)

The third way, dropping Mash's /Mopy and /Data Files folders into /Morrowind works the best for me, and I have experienced far fewer errors than the other two ways. If your Wrye Mash is in a /Wrye Mash folder or other than /Morrowind directory, try moving it here, then retest.

If that doesn't solve it, try a fresh download of WMSA 85, and install as #3 above.

Otherwise, I'll let Yakoby decode the traceback, but I think it's a local problem rather than a code problem.

If you still have problems, can you answer these questions ...

• What OS are you running?
  
• If WinVista or Win7, is UAC enabled or disabled?
  
• If WinVista or Win7, in what directory is /Morrowind installed?
  
• Do you run Mash as administrator?

Good luck,

-RMWChaos

Yeah - huh.

I installed only this version of mash with this latest install.

I used the installed folders provided by the contents of this package within the Morrowind folder.

I do have Wrye Python 03a installed - perhaps there is interference? I need it installed though because i also run Wrye Flash for fallout, and 3 Wrye Bash's for 3 Oblivion installs (2 main games and Nehrim).

Other answers:

No I never install games to default locations - nor do I ever turn off UAC on my Vista 64 system.
I can run MGE XE without admin privileges and without issue.
The games are installed on a completely different drive than the OS and way out of the domain of UAC problem areas.

I await translation of the error too.

@Yakoby ... here, I wrote a book for you! :read:


Ok, installed Python 2.7.1, Unicode wxPython 2.8.12.0, which says is for Py2.7, and comtypes 0.6.2. Even though I'm running a 64-bit OS, I went with the 32-bit versions, assuming this is what most folks have now.

Checked Wrye 84, and it seems to work properly. Also checked to see that it didn't break WMSA 85, and that runs properly, too. So all good there.

Now just need to get Wrye Mash 85 running in python mode, not stand alone, and I should be good to go. Already forked it and cloned it locally.


I really do think it's a good idea. I'd have it output to the /Data Files/tes3cmd/Backups folder. The updated tes3cmd batch file I wrote uses this path as the default for both ESP/M backups and log files, with the "--hide-backups" option (but not the broken "--backup-dir" option) for ESP/M files, and a matching "> [path] / plugin.name-cleaned.log 2>&1" command for log files.

I'd have the backups of the original 'dirty' mods go to the default tes3cmd path as well, which should avoid confusion / overwriting of Mash backups in its default directory /Data Files/Mash/Backups.

So something like this should do the trick, I would think...

Where %cd% is "current working directory"tes3cmd clean --hide-backups --replace [selected files] > %cd%/tes3cmd/backups/[selected files]-cleaned.log 2>&1...follow it up with...tes3cmd header --hide-backups --synchronize [selected files] >> %cd%/tes3cmd/backups/[selected files]-cleaned.log 2>&1...or...tes3cmd header --hide-backups --update-masters [selected files] >> %cd%/tes3cmd/backups/[selected files]-cleaned.log 2>&1...which will prevent those annoying Yellow or Orange unsynced masters check boxes in Mash.Notice the double >> on the second two lines? This appends to the log file created in the first line.

That's the DOS command, anyway; not sure how it would be done in python or the GUI wrapper method you're using.

The thing to note here is that the two lines of code in conjunction will create a "plugin.esp" and a "plugin~1.esp" in the /tes3cmd/backups folder if it cleans the file and updates masters. Not a bad thing, IMHO, but worth noting in help files so that users understand what each one is: "plugin.esp" is the dirty mod and "plugin~1.esp" is the cleaned mod without masters updated.


tes3cmd does have some code that ignores Official ESM files as well as some others; however, dropping the plugins onto a batch file overrides this functionality - I think because it passes the entire path to %1 rather than just the plugin name; so I could easily fix this with %~nx1, but I don't think ignoring them completely is necessarily the right thing to do. We all know gamesas coders made some mistakes and dirty mods along the way. As long as the GMSTs are left alone, it shouldn't cause any problems to clean official masters.

Here's a weird thing, though ... there are supposed to be 72 total 'Evil-GMSTs' - 61 from Bloodmoon and 11 from Tribunal. If you clean Tribunal.esm and Bloodmoon.esm with tes3cmd batch, without the batch checks, it will find only the following...

Tribunal:
5 Tribunal GMSTs
61 Bloodmoon GMSTs

Bloodmoon:
9 Tribunal GMSTs
21 Bloodmoon GMSTs

Huh. Have to figure that out.

Where there is a problem is with mods that use a master and then a "control file" plugin. In some cases, the duplicates in the plugin are intentional to prevent overwriting from other mods, but in other cases, they really should be cleaned. Short of a "no clean" list like John uses in his code, I don't see an easy way around this.

I'm seriously considering taking http://code.google.com/p/mlox/source/browse/trunk/util/tes3cmd?r=697 and porting it to python so that it can be truly integrated into Mash, rather than using the GUI wrapper method.

Doing that would offer a great deal more flexibility in defining what is and isn't cleaned, including opening the door for user-defined lists in a .csv file with flags for "NoGmst", "NoDups", "NoClean", etc.

Also, would be easier to distinguish between dirty backup and clean backup without updated masters by editing the name of the backup file depending on the command.


I tested this, and you're partly correct. When you backup a file with Mash, it creates a name-based copy, but also creates a .ESMF file. If you backup a different mod with the same name, it overwrites the first copy, but the .ESMF file remains the same. The .ESMF file is the "very first automated backup" that Mash makes of the file, and will not change, I assume, unless you delete it.

The help file suggests using Snapshots to create multiple copies of mods, appending -01, -02, etc. to the end of each to distinguish. But you would still have to know that My_Plugin-01.esp was of a different mod, not just a later version, than My_Plugin-02.esp.

Duplicate just creates a copy of the mod in the /Data Files folder, allowing you the option to give it your own name.


As far as the batch file, I figured out most of what I needed to do. What I couldn't do within tes3cmd code, I did with DOS batch commands.

However, I did find that the tes3cmd "--backup-dir" option appears to be broken, even from the command line.

I tried these different commands straight from the /Data Files command line, but none worked...

These...tes3cmd clean --replace --hide-backups --backup-dir "c:\temp\backups" plugin.esp > plugin.esp-cleaned.log 2>&1tes3cmd clean --replace --hide-backups --backup-dir c:\temp\backups plugin.esp > plugin.esp-cleaned.log 2>&1tes3cmd clean --replace --hide-backups --backup-dir "c:\temp\backups\" plugin.esp > plugin.esp-cleaned.log 2>&1tes3cmd clean --replace --hide-backups --backup-dir c:\temp\backups\ plugin.esp > plugin.esp-cleaned.log 2>&1...throw this...FATAL ERROR (): "c:\temp\backups" does not end in .esm/.esp/.ess" at script/tes3cmd line 4736...and these...tes3cmd clean --replace --hide-backups --backup-dir "c:\temp\backups\plugin.esp" plugin.esp > plugin.esp-cleaned.log 2>&1tes3cmd clean --replace --hide-backups --backup-dir c:\temp\backups\plugin.esp plugin.esp > plugin.esp-cleaned.log 2>&1...throw this...FATAL ERROR (): Not a plain file" at script/tes3cmd line 4739

However, just using "--hide-backups" without specifying a backup folder works just fine.

Get back to me when you can. I still have to figure out this GitHub stuff and start looking through the code.

Take care,

-RMWChaos

I ran into problems with WMSA after installing Wrye Python 03a ... works fine on a straight Python 2.7.1 + Unicode wxPython 2.8.12.0 + comtypes 0.6.2, though. I'd venture to guess, based on both our problems we've had, that 03a is the problem, but not sure why.

Do you think Flash and Bash will run with that Python setup instead of 03a?

Best I can offer at this point.

-RMWChaos

I don't think Wrye Flash will - not sure about Bash.

Will have to investigate further later tonight - money has greater pull.

Really I'd just revert to python dependent - it is just that the features of this are the glitter prize. I'm certain I can ask the Bash devs and they'd have an answer.

So far this is the only error.

Although WMSA should, theoretically, be entirely independent of any other python installs, it's the only answer I have for why my copy was acting squirrelly when I had 03a installed.

As I mentioned, it works as designed on my copy of WMSA 85 with Python 2.7 installed; so I don't think it's the code that's broken here, but who knows? :shrug:

Should be able to use the standard python version of Mash 85. Yakoby's posted a link to his public repo where you can download it.

The repo version there is mash.py not the compiled WMSA; so you can give it a try and see what happens.

-RMWChaos

This is sort of implemented. Not really tested yet though.


The 'evil' GMSTs that are added to plugins when they aren't defined in loaded master files are the GMSTs that were created when the developers made the expansions. Some of the 'evil' gmsts have the same values as defined in the expansions (which is why those were cleaned). Others don't. (Which is why they aren't cleaned).


IDK how much benefit it would be over tweaking the Perl code and implementing things in on the Mash side. :shrug: Plus the bit of code that actually calls tes3cmd isn't that complex. Probably ~120 lines and some of that is the threading interface. True about the backups, although that may be fixable on the perl side fairly easily.

Up to you :shrug:


I can't work on this that much atm. My life is full of exams.

Correction - the error I reported above only happens if I first clean the plugin (while Mash is open and the esp is active) then cut and paste it out of the data folder and return to mash.

It does not happen if I just cut and paste without cleaning.

Seems minor - I'm not going to jump on drastic things like updating python over this alone.

I will look into it.

BTW: Any chance of copying issues/feature requests here:
https://github.com/Yacoby/Wrye-Mash/issues

It is easier to manage them and they won't get lost so easily. Also if (or more likely: when) I give up there will be a list for whoever continues.

They'd have to create an account either with WordPress or GitHub to do that, Yako.

We can troll the posts and log them in there instead. Do you want all issues, and you can close out the ones you corrected, or just the new / unresolved ones in there?

-RMWChaos

[edit]
I opened a new issue on GitHub as follows...





That work for you?

@RMWChaos Ta for adding the issue

Just the new and unresolved ones matter.

Psymon:

This?: http://www.uesp.net/wiki/Tes4Mod:Wrye_Bash/Basic#List_Mods.2FMasters


Could you expand on what you mean by this?

I also have listed the py2exe builds in the OP

I think he means the markers " ==Last== " that's greyout in the Installers tab. Not sure though. :unsure:

Yes Markers is what I meant. Sorry forgot name.

Originally they were empty archives in the BAIN folder but at some point were changed into UI features of the BAIN tab.

Of course along with list mods like in the post above (yes what I meant) there is also the list packages command in later versions of bash for the other games.

allows one to do something like - here is the first half of my BAIN packages for Fallout3:They allow for helpful categorization of BAIN archives. In the above they are the ones with == at the start and end ==

It and the list mods allows one to post info about what is or isn't active (mods) or installed (BAIN). And admittedly they are not important bug fixing features - just nice.

What I think would be awesome is drag and drop sorting - that is in Wrye Bash and Wrye Flash.

And thank much for considering my requests and putting up with my posts.

On Mods Tab, just right click on the headings bar (File, Rating, etc.) and click "Copy Active Mods". This will copy all checked mods to your clipboard in load order where you can then paste them, like this...

Active Mods001  Morrowind.esm002  Tribunal.esm003  Bloodmoon.esm004  plugin.esp

-RMWChaos

Makrers were added in a couple of days ago


Ok


All feedback is good feedback


Of course.