Tes4Edit Cleaning Guide on CS Wiki : IV - Oblivion

» Wed Mar 30, 2011 8:08 am

I personally suggest that any opinionated remarks be removed from the wiki for the sake of keeping it factual. Things like "needs serious cleaning" should be removed, considering that it just makes mosd seem like a bad piece of work, which they are not.

I'll get behind that right now.

Edit: Done. Please check if I missed something.
I left the entry for ~~Haunted House~~ "HautedLairMod.esp" as is because I don't know the mod and cannot make sense of the comment, especially the "All mod" part of it.

» Wed Mar 30, 2011 12:38 am

@ Tommy_H: Recompile All means that the mod was saved after the author had hit the "Recompile All" button in the CS, which before shadeMe's CSE, recompiled every single loaded script, which means every single vanilla script gets compiled and saved into your plugin. This can cause absolute mayhem when loading the mod due to all the script overwrites if you have other mods modifying those scripts, so I'd suggest leaving it, if only to let the end user know how bad the Recompile All command is was.

I've put the Mystery of the Dulan Cult entries back in both areas, and clarified which versions are where. Standalone is dirty, Better Cities replacer is clean.

» Wed Mar 30, 2011 9:07 am

I agree - things like "recompile all" should be mentioned because that IS a terrible thing to do and the CS wiki warns against it so there's no excuse for finding it in a publicly released mod.

Besides the obvious mangling of scripts, it could cause CTDs for non-SI users if that happens with a modder who has SI. There are a few vanilla scripts modified by SI with SI specific things in them.

» Tue Mar 29, 2011 9:39 pm

Can someone else check NakedNord.esp to clarify with my/Gurgl's findings?

I believe it was removed from the "To Be CLeaned" list to the "Already Cleaned" clists based on my input, but Gurgl (and Vorians) think it may not be clean. I already might have cleaned my copy earlier, so I may be mistaken and would like someone to confirm.

Also, can someone else confirm if they get some many UDRs and ITMs with the latest Elsweyr. I am using the Nov 2010 addon esp, but Iliana thinks that the mod is already clean. Thanks!

» Wed Mar 30, 2011 5:17 am

Naked Nord.esp: 18 ITM, 0 UDR. Also has a few edits I personally consider undesirable like changing some cell names. One wild dialogue edit missed when the English translation was done. All from the download currently available on Nexus.

I don't recall the status on Elsweyr but I think the November 2010 ESP did have some leftover dirt and deletions that needed fixing.

» Tue Mar 29, 2011 8:06 pm

Naked Nord.esp: 18 ITM, 0 UDR. Also has a few edits I personally consider undesirable like changing some cell names. One wild dialogue edit missed when the English translation was done. All from the download currently available on Nexus.

Ah, thanks you've reminded me what my problem was with Naked Nord! The mod on its own works fine in-game, doesn't cause crashes. But I merged it with some other mods, and the merge failed due to the vanilla cell renames, so I had to remove those cell renames to succeed in the merge. So it has a small number of ITM edits, but otherwise is fine.

» Wed Mar 30, 2011 9:08 am

Thanks for the clarifications, Arthmoor and Vorians. I checked and found that I had already cleaned naked Nord before, and that is why I couldn't find the errors everyone else found.

Can someone check Elsweyr as well. Iliana checked her version, and it doesn't seem to have the ITMs and UDRs I get. Maybe I have the wrong version, but I downloaded the Nov 2010 addon from nexus....

» Wed Mar 30, 2011 8:29 am

Reaper's Waterfront.esp tested:

[Removing "Identical to Master" records done] Processed Records: 2968 Removed Records: 14 Elapsed Time: 00:00
[Undeleting and Disabling References done] Processed Records: 2954 Undeleted Records: 60 Elapsed Time: 00:00

(Note, this is for the standalone mod, and not the BC integrated version)

Wiki list of mods to be cleaned has been updated.

» Tue Mar 29, 2011 11:51 pm

Thanks for the clarifications, Arthmoor and Vorians. I checked and found that I had already cleaned naked Nord before, and that is why I couldn't find the errors everyone else found.

I run into this at times too...can't recall if and when I cleaned a particular esp. What I'm getting in the habit of doing is including both clean and unclean versions of esps as selectable options in my BAIN packages. I sometimes also do this with PyFFi meshes. A typical package might look something like this

00 ESP - TES4edit cleaned
00 ESP - original version
01 Textures and voice
02 Meshes - PyFFi 2.1.7
02 Meshes - original

Doing it this way is a bit more work but works out well in the long run.

» Tue Mar 29, 2011 9:18 pm

Cleanup of Kvatch Rebuilt, version 1.1

http://www.tesnexus.com/downloads/file.php?id=15412
That's the original, "Non-Grammar Patch" version. There is a "Grammar Patch" version (which I am successfully using) that has Tesnexus ID no. 33330. Its cleanup run yields more records processed but the same number of issues found (26 and 4, see below)

What I am reporting here is the cleanup run for the original "Kvatch Rebuilt.esp" v1.1 only. It lists Oblivion.esm and Kvatch Rebuilt.esm as masters.

Found issues:

[Removing "Identical to Master" records done] Processed Records: 2895 Removed Records: 26 Elapsed Time: 00:00
[Undeleting and Disabling References done] Processed Records: 2869 Undeleted Records: 4 Elapsed Time: 00:00

Full cleanup log:

Could you upload the clean version somewhere perhaps? I saw someone cleaned Origin of the Mages Guild aswell.
But more problematic is Knights of the White Stallion as mentioned here: http://cs.elderscrolls.com/constwiki/index.php/TES4Edit_Cleaning_Guide#List_of_Mods_Needing_Cleaning
Would like very much if someone upload cleaned this way version.

» Wed Mar 30, 2011 6:34 am

Could you upload the clean version somewhere perhaps? I saw someone cleaned Origin of the Mages Guild aswell.
But more problematic is Knights of the White Stallion as mentioned here: http://cs.elderscrolls.com/constwiki/index.php/TES4Edit_Cleaning_Guide#List_of_Mods_Needing_Cleaning
Would like very much if someone upload cleaned this way version.

It tends to annoy modders when you re-upload their work without their permission, not to mention it also incurs the wrath of the banhammer on several sites. Besides, it's easy enough to do an automatic cleaning pass to make it not worth the trouble.

» Wed Mar 30, 2011 8:22 am

I got your point. Will try to do it myself I think.
However the case of KotWS is a bit more complicated.

» Wed Mar 30, 2011 8:28 am

I got your point. Will try to do it myself I think.
However the case of KotWS is a bit more complicated.

Apropos KotWS, the cleaning instructions mentioned in the guide is a bit over my head. I'm such a new player. Can someone post a more detailed instructions? Thank ye.

» Wed Mar 30, 2011 4:13 am

Seeing a list of mods to be clean I didn't find info about implicit dependencies. Including that information would be crucial, because if you clean a mod and make it without implicit dependencies, you end up with a screwed mod (that is why I'm afraid of cleaning on my own). And knowing which are implicit dependencies is really hard to tell for an averege user.
Of course ideal would be a cleaned pack of esp's already cleaned by users uploaded on some sharing site. I mean if authors of such mods are absent for good and cannot make a cleaned version on their own (they know their plugins best).

» Wed Mar 30, 2011 1:23 am

No ideal would be for mods to be released pre-cleaned.

What is it you mean by implicit dependencies? Either a mod has a master or it does not.

If so then clean the master first, then the esp. If the esp is dependent on another esp then don't clean it. (and of course don't clean the UOP or overhauls).

I've used the automated utility of TES4edit to do this for what almost 2 years now. Rarely do I feel the need to really inspect what got cleaned out. Once you realize what a duplicate record or deleted record is - you will see it rarely does mater.

» Tue Mar 29, 2011 9:20 pm

Seeing a list of mods to be clean I didn't find info about implicit dependencies. Including that information would be crucial, because if you clean a mod and make it without implicit dependencies, you end up with a screwed mod (that is why I'm afraid of cleaning on my own). And knowing which are implicit dependencies is really hard to tell for an averege user.
Of course ideal would be a cleaned pack of esp's already cleaned by users uploaded on some sharing site. I mean if authors of such mods are absent for good and cannot make a cleaned version on their own (they know their plugins best).

Impractical due to the sheer volume of mods and load order configs out there. If I were to update the page to include all possible implicit dependencies for every mod on that list, I'd be 50 by the time I was done. :cake:

TES4Edit cleaning takes some DIY work, there's only so much we can do for you here.

As for the ideal solution, yes it would be nice, but again, I bring up the issue of permissions. This is never going to happen.

No ideal would be for mods to be released pre-cleaned.

What is it you mean by implicit dependencies? Either a mod has a master or it does not.

If so then clean the master first, then the esp. If the esp is dependent on another esp then don't clean it. (and of course don't clean the UOP or overhauls).

I've used the automated utility of TES4edit to do this for what almost 2 years now. Rarely do I feel the need to really inspect what got cleaned out. Once you realize what a duplicate record or deleted record is - you will see it rarely does mater.

There are several mods out there, mostly compatibility patches, that are not explicitly dependent on one (or several) of the mods they are patching for, but still rely on those mods being there to work properly. UL patches exhibit this behavior quite often; that's an implicit dependency.

» Tue Mar 29, 2011 9:49 pm

Not to sound condescending or something but if the difference between an explicit dependency and an implicit dependency isn't clear enough for you to understand, it may be best to avoid cleaning much of anything until you've got a better handle on what's going on.

» Wed Mar 30, 2011 12:12 am

In terms of cleaning mods, checking for them being clean, etc. Why does it matter whether it is these newfangled terms implicit/explicit or not when it comes to using the cleaning tools of TES4Edit?

Just seems to be confusing the issue to even bring it up if the scope of what is being discussed is cleaning.

If it is implicitly has a master then it will have no impact on checking for or cleaning it with edit and if it does have an explicit master then obviously it will need to be loaded to.

» Wed Mar 30, 2011 10:15 am

In terms of cleaning mods, checking for them being clean, etc. Why does it matter whether it is these newfangled terms implicit/explicit or not when it comes to using the cleaning tools of TES4Edit?

Just seems to be confusing the issue to even bring it up if the scope of what is being discussed is cleaning.

If it is implicitly has a master then it will have no impact on checking for or cleaning it with edit and if it does have an explicit master then obviously it will need to be loaded to.

I believe the CS Wiki explains things quite well:

An explicit dependency is very easy to discern because you can easily check what files it relies upon in Wrye Bash by looking on the right-hand side where Masters are listed. Moreover, TES4Edit will automatically load any explicit dependencies (can be either .esm or .esp files) when you select a mod with explicit dependencies. The only time it won’t do this is if you don’t have a mod installed that the other mod is dependent upon. This would be a major mistake you’ve made. For instance, if you download a patch to fix landscape issues with, say, Verona House Bloodlines, you better have Verona House Bloodlines installed!

An implicit dependency is more tricky because you won’t see the required files listed in Wrye Bash. The most common implicit dependency is when dealing with the many landscape patches by Vorians and some by Arthmoor. The only way to know what implicit dependencies a mod has is to know what the mod does, which can discovered by reading its readme usually. Here again is why understanding what a mod does is crucial. Specifically, you must understand which mods a patch mod is designed to patch!

On top of that:

Example #6: Another Illustration of the Importance of Loading Implicit Dependencies

DuggeDank provides the following example. Assume Mod A is a great mod but it regrettably makes a few silly changes. For instance, Mod A changes the name of the NPC Baurus to JohnnyBoy but does a lot of other really cool things. So Mod B is created in order to fix (patch) Mod A. Therefore, Mod B changes the name of the NPC Baurus back to Baurus in order to get all the changes from Mod A except the name change.

If you were to only load Mod B without also loading Mod A into TES4Edit, then TES4Edit would notice that Mod B's record of for Baurus is identical to its master Oblivion.esm. Hence, automated cleaning would remove the name change in Mod B because it appears to be unnecessary. This would be bad because now you’d have JohnnyBoy running around your game again.

This is why it is critically important that you load both Mod B and Mod A into TES4Edit in the proper order (Mod B loads after Mod A since it modifies Mod A).

But as ElminsterEU points out:

This problem [the JohnnyBoy issue] only happens if you load JUST Mod B and not Mod A into TES4Edit at the time you do the [automated] cleaning.
If both Mod A and B are loaded, then that record in Mod B would be marked as ‘identical to master but conflict winner,’ in which case the cleaning should not touch it.

To prevent the possibility of Mod B being loaded without Mod A being loaded, Mod B should list Mod A in its master list in which case trying to load Mod B into TES4Edit will always load Mod A as well.

Elminster’s final point is aimed toward mod creators. It is best and safest that any time you release a mod that modifies or relies upon another mod(s), you should list the other mod(s) as a master in the header of your mod if possible. This may not be practical in some cases, but in most cases it makes sense as a goal to aim for.

Lastly regarding Example #5, the much preferred solution to this whole problem would be for the original mod creator of Mod A to delete the JohnnyBoy edit. But this may not happen because for many reasons such as the mod author is long gone, likes the JohnnyBoy change, and/or is criminally insane. This is why sometimes a patch needs to contain typically unnecessary edits to counter the undesirable edits.

Why has so much time been spent on discussing cleaning philosophy instead of actual cleaning instructions? The reason is because the majority of problems you can cause by cleaning come from a lack of understanding rather than mistakes in the actual cleaning process. Also, the actual cleaning process is very simple (but tedious).

Furtherfore, here's a quote from Vorians outlining rather well how to tell an implicit dependency from an explicit one and why it is important to load them:

This patch is dependent on Baddy.esp, so if you wanted to auto-clean this ESP, TES4Edit will load Baddy.esp when you load Baddy-EntiusGorge patch.esp. It won't however load xulEntiusGorge.esp as the patch is not [explicitly] dependent on the Unique Landscape ESP. [So the user MUST be sure to tick the UL ESP themselves, when first loading TES4Edit. If you forget to tick the UL ESP, you run the risk of removing IMPORTANT edits when automatically cleaning the mod.]

So, the patch has Baddy.esp as a master, making the patch explicitly dependent on Baddy.esp. The filename, readme and entire purpose of the mod makes it clear that xulEntiusGorge.esp is fairly important for the patch to work correctly. So it is implicit that the mod is dependent on xulEntiusGorge.esp even though there is no master entry in the header record.

This means that it is obvious to anyone who is actually intelligent enough to be safe using TES4Edit in the first place (did I say that out loud?!) that to do it right, you need to have both Baddy.esp and xulEntiusGorge.esp loaded into TES4Edit (and in the correct load order!) for the automated cleaning to be reliable [not mess things up].

Therefore…there is no valid reason to not clean a patch where it is blatantly obvious which other mods it was designed to patch.

Does that help?

» Wed Mar 30, 2011 8:41 am

@Psymon: The distinction is critical for the reasons Thomas just quoted from the wiki. An improperly loaded mod with implicit dependencies can result in a bad cleaning of things that may not actually be dirty.

» Wed Mar 30, 2011 1:13 am

Well learn something new everyday.

I'm certainly glad that Arthmoor and Vorians have made the majority of patches.

I'm also certain that I've not done that - but what mods would that be that I missed ... hmmm?

» Wed Mar 30, 2011 1:55 am

Ahhh, good discussion this is!

To the newcomers (Carlos and the other chap): Really, you should not be afraid to clean your mods! Again a good starting point are the "dirty" and "clean" lists that are supplied on the CS Wiki. Over the last month or so there has been a pretty good effort going on to update and verify them, with Thomas Kaira one of the key contributors.

A Big Thank You for Thomas!

I think Thomas will agree with me when I say that all mods on the "need cleaning" list do not have implicit dependencies and are therefore perfectly safe to clean with the automated process as outlined in the Guide (finding ITM's and UDR's).

I view it this way: Faced with choice of not cleaning your mods at all because of fear to screw up, especially if they are listed as "need cleaning", or do clean and possibly run into an issue with a chance of 1 in 200 (maybe) I'd say DO clean. Not the least because if there should be an issue Tes4Edit always makes a backup copy of the plugin. And you should keep copies of all original mod downloads anyway. So it's always easy to revert to the unclean version.

The process of cleaning as per the Guide is easy and quickly done. It just requires a bit of time, mostly waiting for the PC to finish the processing and filtering. It's perfect for doing on the side while surfing the net...

And then there are always the forums / this thread: if in doubt just ask. Chances are someone before you already has experience with it and can guide you.

So in summary, there is really no reason not to engage in cleaning mods yourself... :grad:

» Tue Mar 29, 2011 11:52 pm

Have no issue doing the auto cleaning, in fact already half way through my mod list. It's the special cleaning instructions for KotWS that is really arcane to me.

300_White Stallion 4.esp Special Cleaning Instructions

Requires custom TES4Edit work to fix errors or it can’t be saved. It's too complicated to list here, but it has to do with editing REFR:00092EC9 so that it uses the Upper version of the door and moving the door into the correct position. Copy the coordinates from REFR:020014BE then delete REFR:020014BE. This also applies to the renamed Feudal Empire version.

While we're talking about White Stallion/Fuedal Empire, this mod also has a LOT of errors in the AI Packages. They currently have no faction assignments and need to be updated. You can open the mod in the Construction Set and go to AI Packages and find all the ones with null parameters. Change those to have the following two conditions:

Target no, GetFactionRank, LeyawiinFaction, >=, 0.00 AND Target no, GetFactionRank, KnightsWhiteStallion, >=, 0.00

Like, huh. So once again, would appreciate if someone can help. Thank ye.

» Wed Mar 30, 2011 9:06 am

Well I probably shouldn't be the one to comment on those instructions and I'd bet my bottom dollar Vorians wrote them.

But I will say this about KotWS and Feudal - it ain't all that. Especially Feudal. It expands one quest with some rewards and a better lodge. After doing the quests I couldn't wait to remove the mod. I also never cleaned it and it played fine.

» Wed Mar 30, 2011 2:01 am

You also don't want to be having the mod reassign coordinates to the vanilla door. It's been ages since I looked at that mod, but the proper course is to get the added door fixed and setting the vanilla one out of the way as a non-persistent object. Otherwise if you uninstall the mod, the vanilla door to the small house will be permanently displaced.