Q: Cleaning Mods with TESPCD

Post » Mon Aug 16, 2010 3:16 pm

Howdy, y'all !! :mohawk:

Got me some questions about cleaning mods "properly" so as not to break nuthin' if you catch my drift.

Here's the situation...

I have more MW mods than I know what to do with. Using Wrye Mash, I've got most of 'em working properly; however, there are of course a few little conflicts here and there.

A few mods tend to break the game outright, for instance "Immersive Chargen v1.25" which gets you stuck with the annoying, "Who's there?" response. While IC v1.24 does not seem to have this issue, I tried cleaning (remove all unclean entries) IC v1.25 with TESPCS v0.31, which surprisingly seemed to fix it without breaking anything else.

So why not clean everything this way? Well, that's the essense of my question. Doing some research around the web, I found a few threads that suggest certain mods should absolutely *not* be cleaned, period. Well now, why is that, I wonder? Here are some examples...

  • Giants

  • Marksman Mod

  • Blood and Gore (?)

  • Any Balance Mod (Wakim’s, Big Mod, etc.)


The entries I'm interested in are the ones that say they are identical to the original entries in Morrowind.esm, Tribunal.esm, and Bloodmoon.esm.

So my question is thus...

What is the purpose of entries identical to those in the ESM files in mods such as Morrowind Patch Project (MPP) v1.6.5b? Are they supposed to be there, or are they truly unclean entries that should be cleaned?

Some might respond, clean them and see if it breaks anything, to which I respond, how would I know short of the game crashing where it hadn't before?

You see my quandry.

Thanks in advance for your responses and help.

Regards,

-RMWChaos
User avatar
Laura Ellaby
 
Posts: 3355
Joined: Sun Jul 02, 2006 9:59 am

Post » Mon Aug 16, 2010 9:37 pm

I highly recommend using http://code.google.com/p/mlox/wiki/Tes3cmd, it will do a really through job of cleaning away the duplicate entries (and some other stuff).

Here's a little how-to threw together quickly a while back.
Spoiler


TES3CMD by John Moonsugar

This program is great at cleaning, fixit, fiddling with a mod. It's very thorough and catches stuff other cleaner do while not breaking the mod (Yes, I'm looking at you TESTOOL). It can update the headers to match the ESMS, recover bad mods, dump data, and so much more. It even has a "fixit" command that will actually fix it (unlike ...ahem cough..TESTOOL...cough).

The main drawback is no GUI. That does limit audience, because to be honest while versatile, the command line just isn't that much fun. Heck in windows, its a pain to even get it the directory you want. But I have a way:

Manual Method:
Find your Morrowind/DataFiles directory
Right click it while holding shift
Select "open cmd window here" and a Command box will appear with the directory all ready set to Data Files.
As long as TES3CMD is in that directory,
Type "TES3CMD clean " or "Tes3cmd fixit" (to fix everything)

I also have made a bat file that will allow you to drag and drop an esp right on top of it and it will clean it then put the results and log file right back where the ESP came from.
I use this for my "Morrowind Archive" directory. I like to clean the mods before I put them in my Data Files directory. Here's how (sampled for TESELINT)

More "I hate command lines" friendly method:
Open Notepad. Cut and paste below. Save as .bat file where ever you installed TES3CMD. Then find the file, right click and select Send->Send to desktop as shortcut. Now you can drag an esp from any folder onto your desktop shortcut and it will clean it and put the clean one right back where it came from.

Batch file text (cut and paste the below)
@echo off
rem drag and dropable bat file
rem put shortcut on desktop and drag esp onto it
rem "clean" file and log will be added to same folder the esp came from

tes3cmd clean %1 > %1-cleaned.log
type %1-cleaned.log
pause

In my experience 95% of my mods were somekind of dirty. And this fixes that.


TESPCD is good as a diagnotic tool but I don't directly clean with it . Also, I don't believe it updates the record count in mods that it edits. And I advise against using it to delete any dialogue. It doesn't "see" the PNAM and NNAM entries the surround the info entries.
User avatar
Marilú
 
Posts: 3449
Joined: Sat Oct 07, 2006 7:17 am

Post » Tue Aug 17, 2010 12:32 am

TESPCD is good as a diagnotic tool but I don't directly clean with it .

This.

TESPCD will break a lot of mods. Try Darknut's GDR and you will see. I never had problems with tes3cmd with exception of a tiny issue: It updates the timestamp in some cases. Nothing major though.
User avatar
Add Meeh
 
Posts: 3326
Joined: Sat Jan 06, 2007 8:09 am

Post » Tue Aug 17, 2010 3:40 am

Yea, TES3CMD owns. Hey, does anyone know if .37 is stable? Does .37 clean things any better if that is the only thing one is using it for?
User avatar
{Richies Mommy}
 
Posts: 3398
Joined: Wed Jun 21, 2006 2:40 pm

Post » Tue Aug 17, 2010 2:22 am

Yea, TES3CMD owns. Hey, does anyone know if .37 is stable? Does .37 clean things any better if that is the only thing one is using it for?


I have only ever used the .37 stand alone version so I can't say if it cleans more or not. Might just be .37 because of it being "standalone". I'm very happy with it though.
User avatar
*Chloe*
 
Posts: 3538
Joined: Fri Jul 07, 2006 4:34 am

Post » Tue Aug 17, 2010 12:14 am

Looking at the instructions above for creating the bat file method:
Open Notepad. Cut and paste below. Save as .bat file where ever you installed TES3CMD. Then find the file, right click and select Send->Send to desktop as shortcut. Now you can drag an esp from any folder onto your desktop shortcut and it will clean it and put the clean one right back where it came from.
a couple of questions:

1. Do we still keep the tes3cmd main program thing in the data files?
2. Will I need admin privileges to use that method?

In trying I did keep the tes3cmd and newly created tes3cmd.bat files in the data folder and pulled an esm (MCA.esm from my archives directory) over to clean and it opened a cmd shell then gave me a message saying it was not a proper command. I then moved the MCA.esm over to the data folder and tried it and a shell window has been open now for 10 minutes with a flashing cursor and no text. Like it is hanging - I finally closed that. Same results with the shorcut to desktop method.

the log it created is blank. I tried with a few other mods same result.

My computer is not slow - for instance it only takes a second to clean an Oblivion esm/esp with tes4edit.

How long should I expect?

=====
[edit] err wait never mind my edit.
User avatar
JaNnatul Naimah
 
Posts: 3455
Joined: Fri Jun 23, 2006 8:33 am

Post » Mon Aug 16, 2010 4:36 pm

Looking at the instructions above for creating the bat file method:
a couple of questions:

1. Do we still keep the tes3cmd main program thing in the data files?
2. Will I need admin privileges to use that method?

In trying I did keep the tes3cmd and newly created tes3cmd.bat files in the data folder and pulled an esm (MCA.esm from my archives directory) over to clean and it opened a cmd shell then gave me a message saying it was not a proper command. I then moved the MCA.esm over to the data folder and tried it and a shell window has been open now for 10 minutes with a flashing cursor and no text. Like it is hanging - I finally closed that. Same results with the shorcut to desktop method.


The tes3cmd file should be in the data file directory and so should the bat file. I should note that I use the .37 standalone version, not sure if that matters though. I shortcut the bat file the desktop so I don't have to open my data files directory. Tes3cmd creates a cache file of ESMs to reference when it cleans ESPs. It will then reference those ESMs whenever a mod references them and clean based on that. The tes3cmd directory is where they are stored. You could delete the cache files and it will regenerate them (this has worked once for me). If MCA is dependent on any ESMs they need to be in the data file directory too but it should work then.


I'll download it and see if it works for me.
User avatar
Luis Reyma
 
Posts: 3361
Joined: Fri Nov 02, 2007 11:10 am

Post » Mon Aug 16, 2010 11:48 pm

Ahh - oh so there is a required program - perl - didn't see that. yeah I was trying with .36 and without the perl thing.

Well how do I get .37 working? Just looks like code to me.
User avatar
RaeAnne
 
Posts: 3427
Joined: Sat Jun 24, 2006 6:40 pm

Post » Tue Aug 17, 2010 12:26 am

OK so I installed the stawberry perl thing - checked that it is in the cmd line as per wiki page.

Still getting long 10 minute hanging of cmd prompt. This time however the task manager is showing activity - but now not even output of a log of any sort.

This is using the CMD shell method.

Is this really the best option for cleaning mods with morrowind? really? A program without even a GUI or exe?

Where are those people again who keep saying that modding morrowind is easier that Oblivion?
User avatar
Siidney
 
Posts: 3378
Joined: Fri Mar 23, 2007 11:54 pm

Post » Mon Aug 16, 2010 10:09 pm

Ahh - oh so there is a required program - perl - didn't see that. yeah I was trying with .36 and without the perl thing.

Well how do I get .37 working? Just looks like code to me.


Not sure what you mean, If you download the .37 zip from the link above, the only file inside is "TES3CMD.exe"...dump it in your data file folder and it should work (hopefully).

EDIT: I tried MCA.esm myself and it worked fine. Just a couple of seconds. Not sure whats going wrong for you :(

EDIT2: The .37 version doesn't use Perl so maybe it doesn't work with .36. I remember also someone listing a script for Wyre Mash that would let you do it inside that program, but I don't use it because I didn't want to be tied to Mash just for cleaning a mod. I agree the lack of a GUI is a bummer, and deters people from using a great tool.
User avatar
adame
 
Posts: 3454
Joined: Wed Aug 29, 2007 2:57 am

Post » Tue Aug 17, 2010 5:00 am

OK dang it - I must be near blind today - there is it on the download page - I clicked the svn link which gave me the code confusion.

Testing ...

... when using the bat file - should I expect to see any text in it?

====
[edit]OOP there is it is:
Output saved in: "F:\Bethesda Games\Morrowind\Data Files\Clean_MCA.esm"Original unaltered: "F:\Bethesda Games\Morrowind\Data Files\MCA.esm"Cleaning Stats for "F:\Bethesda Games\Morrowind\Data Files\MCA.esm":       duplicate object instance:     4             redundant CELL.WHGT:   350


But I don't want to place shorcut on desktop - need to think about where.

Thank you very much Pinkertonius.
User avatar
Chase McAbee
 
Posts: 3315
Joined: Sat Sep 08, 2007 5:59 am

Post » Mon Aug 16, 2010 3:13 pm

OK dang it - I must be near blind today - there is it on the download page - I clicked the svn link which gave me the code confusion.

Testing ...

... when using the bat file - should I expect to see any text in it?

First: There is a exe version of tes3cmd 0.37o. You don't need to install perl.
http://code.google.com/p/mlox/downloads/detail?name=tes3cmd-0.37o-2010.08.20.7z&can=2&q=

Then use the command prompt to clean a mod with this command:
tes3cmd clean -R --hide-backups YOURPLUGIN.esp


Then synchronize the header to prevent the annoying version mismatch error:
tes3cmd header --synchronize --hide-backups YOURPLUGIN.esp


or just use the fixit function. It will clean all your mods, synchronize all headers and to do other useful fixes, like the NVIDIA fog fix and a leveled list merge:
tes3cmd fixit


the --hide-backups parameter moves all backups to the tes3cmd/backups folder instead of leaving hundreds of PLUGIN~1.esp files in the Data Files folder.
User avatar
kelly thomson
 
Posts: 3380
Joined: Thu Jun 22, 2006 12:18 pm

Post » Tue Aug 17, 2010 2:09 am

Synchronizing headers? wha?

I hope this is not something that requires all plugins you are going to use be installed at once.

So then normally with Oblivion I like to clean the plugins prior to packaging into BAIN packages and installing.

Are you suggesting that it is better to clean plugins only after all that you are going to use are installed. That I don't like as it creates a messy BAIN tab in Wrye Mash.

===================

I uninstalled the perl strawberry - and yet I still see it in the cmd prompt when I run echo %PATH%
User avatar
tegan fiamengo
 
Posts: 3455
Joined: Mon Jan 29, 2007 9:53 am

Post » Mon Aug 16, 2010 3:25 pm

Synchronizing headers? wha?

I hope this is not something that requires all plugins you are going to use be installed at once.

It will only prevent this annoying message at the start of the game.

One of the files that "PLUGIN.esp" is dependent on has changed since the last save.
This may result in errors. Saving again will clear this message
but not necessarily fix any errors.
One or more plugins could not find the correct versions of the master files they depend on. Errors may occur during load or game play. Check the "Warnings.txt" file for more information.


So then normally with Oblivion I like to clean the plugins prior to packaging into BAIN packages and installing.

Are you suggesting that it is better to clean plugins only after all that you are going to use are installed. That I don't like as it creates a messy BAIN tab in Wrye Mash.

I am usually doing the same. I clean the plugin, repack it and install it with BAIN.

I uninstalled the perl strawberry - and yet I still see it in the cmd prompt when I run echo %PATH%

Environment variables only updates when you log out and log back in.
User avatar
Prisca Lacour
 
Posts: 3375
Joined: Thu Mar 15, 2007 9:25 am

Post » Tue Aug 17, 2010 4:21 am

....the --hide-backups parameter moves all backups to the tes3cmd/backups folder instead of leaving hundreds of PLUGIN~1.esp files in the Data Files folder.


This one of the reasons I use the method I listed. Since it copies the "clean" esp/esm back to the directory you dragged it from. It doesn't clutter up the data files folder and I (like you two) can package it THEN install it.

It'd be very simple to add "--synchronize" (or any other arguments you desire) after "--clean" to the bat file to have it do that too I suppose. My method is designed to avoid the cmd line since most people don't feel comfortable using it.
User avatar
Etta Hargrave
 
Posts: 3452
Joined: Fri Sep 01, 2006 1:27 am

Post » Mon Aug 16, 2010 6:36 pm

OK - well since I don't plan to add anything without first cleaning it then the synchronize option wont be needed as much - maybe with plugin updates.

Question - without the hide command - does that mean the esp that is in the data folder is the cleaned one?

I think at this point I will just be moving plugins (and their masters and dependents) to the data folder - clean from there then copy back out to my archives. I have to do this with tes4edit so am very used to this kind of procedure.

... Oh wait I see Pinkertonius' reply above ... so that is in the bat file - goody.

... thinking though - couldn't those extra commands be added to a shortcut ... I guess not if a cmd shell has to be run. Oh well.

I may not be able to figure out to get a RAID set up on my motherboard, but I learned this today. So that feels better.
User avatar
Gracie Dugdale
 
Posts: 3397
Joined: Wed Jun 14, 2006 11:02 pm

Post » Mon Aug 16, 2010 2:50 pm

Wait here is something I'm not found of - renaming plugins.

Can that be disabled?

Seems both methods does that.

==========

With tes4edit - the idea is to clean masters first then the mods that depend on them.

Is that true with Morrowind too?

I ask because if I clean for instance MCA.esm then the plugins that depend on it - wont they be referencing MCA.esm and not Clean_MCA.esm?
User avatar
brian adkins
 
Posts: 3452
Joined: Mon Oct 01, 2007 8:51 am

Post » Mon Aug 16, 2010 8:38 pm

Wait here is something I'm not found of - renaming plugins.

Can that be disabled?

Seems both methods does that.


No. My method does not do that. The -R switch forces tes3cmd to replace the old .esp file.
User avatar
Etta Hargrave
 
Posts: 3452
Joined: Fri Sep 01, 2006 1:27 am

Post » Mon Aug 16, 2010 2:42 pm

OK I edited my post above with this section:
With tes4edit - the idea is to clean masters first then the mods that depend on them.

Is that true with Morrowind too?

I ask because if I clean for instance MCA.esm then the plugins that depend on it - wont they be referencing MCA.esm and not Clean_MCA.esm?


Can I change the Bat file to do the --R bit?
User avatar
Kaley X
 
Posts: 3372
Joined: Wed Jul 05, 2006 5:46 pm

Post » Mon Aug 16, 2010 5:20 pm

OK I edited my post above with this section:


Can I change the Bat file to do the --R bit?


Yes you can add any of the command line arguments to the bat file and it will run the same as if it were on the command line. Also, if you don't want the log file...just delete the ">filename.log" part
User avatar
Andrew Lang
 
Posts: 3489
Joined: Thu Oct 11, 2007 8:50 pm

Post » Mon Aug 16, 2010 11:49 pm

By the way: It's -R, not --R. Parameters with one dash are always shortcuts. --replace is the full version.
User avatar
Alex Vincent
 
Posts: 3514
Joined: Thu Jun 28, 2007 9:31 pm

Post » Mon Aug 16, 2010 9:11 pm

Yes you can add any of the command line arguments to the bat file and it will run the same as if it were on the command line. Also, if you don't want the log file...just delete the ">filename.log" part

Sorry I got sidetracked ... could you tell me what to either add or remove from the bat file to get that to work.

I've never really edited a bat file and not sure where that would go.

I don't even see the part with ">filename.log" part as you wrote above.

thanks

=========

[edit] then also - is my concern about the masters of esp not having the same name really an issue.

Is it like with Oblivion (which I'm far far more familiar with) where if an esp has a master (whether esm or esp) it does so because it reference records in those plugins and masters - therefore by cleaning a master and having it renamed then cleaning the dependent record - the older non-cleaned version would be referenced instead of the cleaned one.

Or is what is being cleaned not those kinds of records so it doesn't matter anyway.

Is there a break down somewhere about what kinds of records need to be cleaned or are the most problematic

thanks again.
User avatar
Eric Hayes
 
Posts: 3392
Joined: Mon Oct 29, 2007 1:57 am

Post » Mon Aug 16, 2010 9:22 pm

Sorry I got sidetracked ... could you tell me what to either add or remove from the bat file to get that to work.

I've never really edited a bat file and not sure where that would go.

I don't even see the part with ">filename.log" part as you wrote above.

thanks

=========

[edit] then also - is my concern about the masters of esp not having the same name really an issue.

Is it like with Oblivion (which I'm far far more familiar with) where if an esp has a master (whether esm or esp) it does so because it reference records in those plugins and masters - therefore by cleaning a master and having it renamed then cleaning the dependent record - the older non-cleaned version would be referenced instead of the cleaned one.

Or is what is being cleaned not those kinds of records so it doesn't matter anyway.

Is there a break down somewhere about what kinds of records need to be cleaned or are the most problematic

thanks again.

Sorry last post was hurried....

Edit it in Notebook and replace:
tesclean clean %1 > %1-cleaned.log
with
tesclean clean --replace %1 > %1-cleaned.log


If you don't want it to write a log delete "> %1-cleaned.log"

This method will save the cleaned mod with the original name and backup the original with the same name adding "~1" at the end.
(I tested it first and it works)

renaming the master wouldn't necessarily break the game (it'd error though) since all the required records would still be loaded but I wouldn't recommend it anyways. For ESMs, I always try and have the original name. BUT for cleaning, TES3CMD needs the correct named ESM to be in the DATA files in order to clean any esps that are reliant on it. For example, if you want to clean any of the extra MCA related esps, MCA.ESM would need to be in you data directory (at least long enough to TES3CMD to make a cache file copy of it).

EDIT: To address the second part of your question (kind of), cleaning consist of removing duplicate entries from an ESM file. The CS is really bad about "throwing in a little extra" when you're making a mod. Also, sometimes when a modder just view something or clicks on it but doesn't change it, it still adds it. Then there are things like the CELL water height and CELL Ambient light level that it will "fill in" with the default value if it doesn't have one. The biggest problem with these are when mods begiin override one another. It's this kind of duplicate data that cleaning a mod fixes. It is possible to manually clean a mod much deeper than anything automated could (I've done it) but it's alot of grueling anolysis and meticulous tracking/deleting.

Oh, and by far dialogue is the most trickiest, which is why most "dialogue cleaning" is limited to just duplicated entries.
User avatar
Shelby McDonald
 
Posts: 3497
Joined: Sat Jan 13, 2007 2:29 pm

Post » Tue Aug 17, 2010 4:01 am

I use TESPCD to clean my mods.
User avatar
Roy Harris
 
Posts: 3463
Joined: Tue Sep 11, 2007 8:58 pm

Post » Mon Aug 16, 2010 7:49 pm

I use TESPCD to clean my mods.


I used to use it for quite a few years before finding TES3CMD and can't blame anyone for using it. It's just now I feel TES3CMD is a better, safer tool (if only it had a GUI) and cleans deeper and fixes more problems.
User avatar
Mistress trades Melissa
 
Posts: 3464
Joined: Mon Jun 19, 2006 9:28 pm

Next

Return to III - Morrowind