[WIP] New merge objects program : III - Morrowind

» Mon Mar 14, 2011 10:02 pm

Good afternoon.

Yes, I probably will do this at some point. Currently though, "correctness" is a higher priority than "speed". I would like everything to be working before I complicate the program logic further with something like this. Beware premature optimization.

I would, for what it's worth, much prefer correctness over speed. A fast job done poorly is useless. A good job done slowly can be run while I'm away from the computer doing other important tasks. Much as it may grate sometimes, Sleep > Morrowind, leaving 6+ hours a night when my PC could be doing a "slow" job merging.

Rochndil, who also has to cook and stuff...and walk dogs.

» Mon Mar 14, 2011 9:12 pm

I think someone misunderstood me. I prefer (like the above poster) correctness over speed but the problem is that Beta 5 finished the merge records job in 2-3 minutes and beta 6 was finished after ~10 minutes and produced tons of unnecessary records in the esp file. Also the testool Merged_Objects.esp file was ~100 kb large and the esper file over 2 mb.

» Mon Mar 14, 2011 8:04 pm

Hi, I'm trying too make a merge object esp file using a txt file where the two mods filenames are listed. But I can't figure how it works. I'm using this command under Windows XP and my Morrowind is installed in the default directory (eg. C:\Program Files\Bethesda Softworks\Morrowind\)

esper -b pluginlist.txt -r

I have also tried too use this command format:

esper -b -r pluginlist.txt

, but that only output an empty output.esp file.

My text file look like this (pluginlist.txt):

Better Clothesv11.espPiratelords Trade Enhancements.esp

I can't understand what's wrong here. Can anyone help me?

» Mon Mar 14, 2011 10:18 pm

Just stumbled across this thread. A new tool for merging objects would be great indeed, since (as the OP says) TesTool has a number of issues which will probably never be solved since we don't have the source.

One request: Make any buffers, arrays, memory assignments etc. adjustable or at least *really* large. One of the biggest problems with TesTool is that it can only handle so much data - once you've rached a certain threshold, it will crash. It's not a certain mod or combination of mods that make it crash (as can be easily verified by removing the mods for which the crash occurs and see it crashing on the next mods), it's just that the author chose a fixed limit somewhere which he thought to be big enough, but which definitely isn't any more.

» Mon Mar 14, 2011 11:12 pm

Hi, I'm trying too make a merge object esp file using a txt file where the two mods filenames are listed. But I can't figure how it works. I'm using this command under Windows XP and my Morrowind is installed in the default directory (eg. C:\Program Files\Bethesda Softworks\Morrowind\)

esper -b pluginlist.txt -r

I have also tried too use this command format:

esper -b -r pluginlist.txt

, but that only output an empty output.esp file.

My text file look like this (pluginlist.txt):

Better Clothesv11.espPiratelords Trade Enhancements.esp

I can't understand what's wrong here. Can anyone help me?

Hmm, can't get -b switch to work either. Try:

esper -r "Better Clothesv11.esp" "Piratelords Trade Enhancements.esp"

Defining the filenames in the esper command works for me, it generates an output.esp as expected.

I'll do some more testing soon.

» Tue Mar 15, 2011 2:24 am

Sorry for the double post but this one is substantially different to the above, and quite a large one.

First off something on the organisation of the readme:

******************************
Options that specify an ACTION
(only one allowed at a time)
******************************

--output , -o
Save output to . By default this is 'output.esp' in the
current directory.

It says "only allowed one at a time" but in the usage examples section this is shown as modifying the "-r" option (esper -r plugin1.esp plugin2.esp -o merged-objects.esp), should this be in the "Options that modify esper's behaviour" section?

Onto the testing, where I put something in QUOTE tags its a quote from the readme and all following text refers to that command. Until the next QUOTE. I've put commands and their output I've used in CODE or CODEBOX tags:

--merge-plugins, -p
Mode = merge plugins

The command

esper -r LeftGloves_Addon_v2.esp "Clear Glass.esp"Esper, version Beta 6Released under GPL; see the file LICENSE for details.Command line arguments received:    ("-r" "LeftGloves_Addon_v2.esp" "Clear Glass.esp")NIL Clear Glass.espLeftGloves_Addon_v2.esp Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #   #>Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #   #>..Merged records:"ARMO": "glass_pauldron_left""ARMO": "glass_pauldron_right""TES3": "LeftGloves_Addon_v2.esp""TES3": "LeftGloves_Addon_v2.esp"

Perhaps add something into the readme about needing to enclose plugins with spaces in the name in speechmarks? But then that's basic MS-DOS syntax so...

Anyway, plugin masters seem to be set right here (Left Gloves is dependent on all three Bethesda ESMs), all are 1234 bytes still.

TES3 records are from the file header aren't they? Not sure if it's worth reporting in the output....

Still seem to have the same issue that the original record from the relevant ESM (if applicable) isn't used as the Original record so I'm not seeing a true merge of the records. Some discussion about that, above. I still think to get a properly functioning merge program there needs to be a compare against the Beth ESMs (Tetchy's idea of cribbing the TESTool DAT file seems to be a good one, guess that'd speed things up...)

Seems to be ignoring the Clear Glass.esp, when dealing with objects affected by both plugins:

Merged records:Original:#   #   #   #   #   #   #>New:#   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #>

When an object is affected by just Clear Glass it's OK:

Original:#   #   #   #   #>New:#   #   #   #   #>Merged:#   #   #   #   #>

And when affected by Left Gloves on its own it's fine too:

Merged records:Original:#   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #>

Order of arguments seems to matter here. This fails:

Merged records:Original:#   #   #>New:#   #   #>Merged:#   #   #>..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Before the above extract from the ESTXT are all the other SKIL records, "Original" is always about Block skill but New: and Merged: seem to be the relevant skill

I guess the ".............." is an indicator of prrocessing records but not finding anything to merge.

As a note, after running this command Output.esp doesn't include the objects from the expansions, however Left Gloves addon includes Tribunal and Bloodmoon items, Clear Glass items just from Morrowind.

--list-active, -l
Mode = list active plugins and masters in Morrowind.ini

Odd that filelist is prefigured with a "T"

I:\Bethesda Softworks\Morrowind\Data Files>esper -l --debugEsper, version Beta 6Released under GPL; see the file LICENSE for details.Command line arguments received:	("-l" "--debug")T I:/Bethesda Softworks/Morrowind/Data Files/Morrowind.esmI:/Bethesda Softworks/Morrowind/Data Files/Tribunal.esmI:/Bethesda Softworks/Morrowind/Data Files/Bloodmoon.esm<snip>

In fact that prefix, "T" here, seems to vary. There's a NIL up in one of the above.

Also the list of plugins contains the full path for each plugin, I'm not sure if that's necessary. Hardly a deal breaker but if the path can be suppressed then that'd be nice.

Checked the active plugin list Esper generates vs that from Mash and they match. Cool.

--check-resources
Scans plugins/masters and looks for all resource files that they depend
on (icons, textures, meshes etc). Then checks to see if those resource
files exist, either on HD or in any of the active BSA files.
After scanning has finished, prints a list of all files that could
not be found.

Still seem to be having the BSA problems. Also it looks like where a NIF refers to a *.TGA file and the texture file's been replaced with a *.DDS then the Check Resources doesn't find the DDS. Maybe suppress the file extension when checking for textures (and/or icons as some are distributed as DDSs)?

--find-conflicts
Follows similar logic to merge-records, but instead of actually
merging anything, prints a list naming all records which are modified
by > 1 plugin in the file list, along with details about which
files are implicated.

Looks cool:

esper --find-conflicts --debug LeftGloves_Addon_v2.esp "Clear Glass.esp"Esper, version Beta 6Released under GPL; see the file LICENSE for details.Command line arguments received:	("--find-conflicts" "--debug" "LeftGloves_Addon_v2.esp" "Clear Glass.esp")T Clear Glass.espLeftGloves_Addon_v2.esp ..Conflict report:Record [ARMO] "glass_pauldron_left":	"Clear Glass.esp"	"LeftGloves_Addon_v2.esp"Record [ARMO] "glass_pauldron_right":	"Clear Glass.esp"	"LeftGloves_Addon_v2.esp"

**************************************
Options that modify esper's behaviour
(any combination of these can be used)
**************************************

--mwdir , -d
"Base" Morrowind directory (where Morrowind.ini and Morrowing.exe
live) is (default is the *parent* directory of the current
directory)

Moved ESPER to root of I:

I:\>esper -l -d "I:\Bethesda Softworks\Morrowind"Esper, version Beta 6Released under GPL; see the file LICENSE for details.Command line arguments received:	("-l" "-d" "I:\\Bethesda Softworks\\Morrowind")NIL I:/Bethesda Softworks/Morrowind/Data Files/Morrowind.esmI:/Bethesda Softworks/Morrowind/Data Files/Tribunal.esmI:/Bethesda Softworks/Morrowind/Data Files/Bloodmoon.esm<snip>

That works nicely

Hope this helps you, excellent work so far :thumbsup:. Hope you continue

[Edit: I knew there'd be some mistake or other in there]

» Tue Mar 15, 2011 3:14 am

Still seem to be having the BSA problems. Also it looks like where a NIF refers to a *.TGA file and the texture file's been replaced with a *.DDS then

eeeickythump already fixed this one, look at bitbucket: http://bitbucket.org/eeeickythump/esper/changeset/91199037b0d0/

» Tue Mar 15, 2011 5:20 am

eeeickythump already fixed this one, look at bitbucket: http://bitbucket.org/eeeickythump/esper/changeset/91199037b0d0/

Oh. I'm using the stand-alone (i.e. no Perl install needed) download. From what I saw, none of the above is fixed in that download. Good to hear the latest SVN picks that up.

» Tue Mar 15, 2011 4:00 am

Hi everyone
I took a break from esper for a couple of weeks. However I have just done some more work on it and uploaded an up-to-date executable with a few more bugfixes.

I am also pleased to report that:
* I merged records from 50 or so plugins into a single output.esp, and Morrowind starts up and loads this file without producing any warnings
* I started a new game using the esp and it appears to work.

Thanks everyone for your testing, especially Dragon32.

The new zip file is esper7.zip, available in the usual place.

Now onto specific points raised in recent posts.

@FallenWizard:
Could the slowdown you noticed be because esper now processes master files (i.e. Morrowind.esm etc) whereas I think earlier it did not?

@Stacet:
The problem you were having with batch files should be fixed. The problem was that the lisp runtime was recognising --batch and -b as options to itself and so failing to pass them to esper. Solved by renaming the option to --batchfile.

@Psyringe:
Don't worry, there are no memory limitations in this program. That really only arises with programs written in C/C++, due to the low-level way that those languages handle memory allocation/deallocation.

@Dragon32:
* The strange problem with SKIL records being merged is fixed.
* As mentioned, check-resources now looks for a DDS file if a TGA file is not found.
* Filenames of plugins/masters need to go AFTER everything else in the command line (docs updated to make this clear)
* The lone "T"s and "NIL"s were some debugging output, now gone.

Still seem to have the same issue that the original record from the relevant ESM (if applicable) isn't used as the Original record so I'm not seeing a true merge of the records. Some discussion about that, above. I still think to get a properly functioning merge program there needs to be a compare against the Beth ESMs (Tetchy's idea of cribbing the TESTool DAT file seems to be a good one, guess that'd speed things up...)

Could you perhaps post what you believe the merged record should contain? I still think it is behaving correctly. A merge where there are only 2 copies of the record is equivalent to just overwriting the older copy with the newer. You need 3 or more copies of the record for fine grained record merging to be useful.

» Tue Mar 15, 2011 2:55 am

Cracking. I did fear that you'd abandoned this. Glad to see you haven't

Latest Esper executable downloaded. I'll post results tomorrow. Changelog looks good.

» Mon Mar 14, 2011 9:22 pm

Geat to see you're still working on this!

Could you perhaps post what you believe the merged record should contain? I still think it is behaving correctly. A merge where there are only 2 copies of the record is equivalent to just overwriting the older copy with the newer. You need 3 or more copies of the record for fine grained record merging to be useful.

I haven't followed the discussions above (and only have a couple of minutes right now), but here are some general thoughts about merging (note that I also may get things wrong because I'm doing this from memory and I haven't worked with these things for a year):

As an example, let's take barrel_01 (an object present in Morrowind.esm). Let's say we have:

Morrowind.esm - contains original barrel_01
P1.esp - changes the mesh of the object to mesh1
P2.esp - adds script1 to the object
P3.esp - changes the mesh to mesh2

The result of an object merger should then be barrel_01 with mesh2, and with script1 attached. I haven't checked your program yet, but I'm certain I'm not telling you anything new here.

However, there are cases which are more complicated. Some details that I remember:

- Some changes need to be done on the sub-record level. For example, IIRC (might be wrong on this) weapon store their damage, value, and weight data in the same record (WPDT). So, if you have one mod that updates the weights of the weapon objects, and another one that updates their value, and perhaps a third one that updates their damage, then the object merger can't simply choose the most recent WPDT record - in this case, the information from the first two plugins would be lost. Instead, the merger would need to break WPDT up into its subparts and compare/pick those. (I believe TESTool does this.)

- Some changes would ideally be accumulative instead of replacing each other, e.g. changes to the contents of containers. If one mod adds five gold coins to the contents of barrel_01, and another one adds two Salt Rice, then the merging program would ideally produce a record which includes both. I don't remember right now how the Morrowind engine or TESTool deal with this.

- Some plugins are patches, and are meant to overwrite the objects of their parent files completely. It might be useful to give the merging program an "exception list" of mods which are meant to be used as pure replacers (might also be overly complicated though).

- Most documentation states that esms can't modify objects from other esms. This is wrong, it's a limitation of the Construction Set which has been misunderstood as a limitation of the engine. For an object merger, it's probably best to just treat esms and esps identically except that esms come first in load order (not sure what your program does).

I'm sorry about the disorganized and half-informed manner I'm giving information here.

But I figured that it wouldn't hurt to at least try to provide the details I remember. I'd probably need a week to get back to a state where I actually "know" these things, and my plate is rather full at the moment ...

» Mon Mar 14, 2011 3:00 pm

@FallenWizard:
Could the slowdown you noticed be because esper now processes master files (i.e. Morrowind.esm etc) whereas I think earlier it did not?

Yes, this is the culprit. I tagged Morrowind.esm, Tribunal.esm and Bloodmoon.esm with {{ESPER:NOMERGE}} and it became a lot faster.

» Mon Mar 14, 2011 7:48 pm

- Some changes need to be done on the sub-record level. For example, IIRC (might be wrong on this) weapon store their damage, value, and weight data in the same record (WPDT). So, if you have one mod that updates the weights of the weapon objects, and another one that updates their value, and perhaps a third one that updates their damage, then the object merger can't simply choose the most recent WPDT record - in this case, the information from the first two plugins would be lost. Instead, the merger would need to break WPDT up into its subparts and compare/pick those. (I believe TESTool does this.)

Yup, esper processes recurses through all subrecords for each record, and through all slots (member variables) for each subrecord.

- Some changes would ideally be accumulative instead of replacing each other, e.g. changes to the contents of containers. If one mod adds five gold coins to the contents of barrel_01, and another one adds two Salt Rice, then the merging program would ideally produce a record which includes both. I don't remember right now how the Morrowind engine or TESTool deal with this.

Esper does take this into account. The function "merge-method" chooses the method of record merging that should be used for each record/subrecord combination.
The methods :append-multiple and

verwrite-multiple do this "accumulative" merging you talk about. Whether it works correctly is another matter...

- Some plugins are patches, and are meant to overwrite the objects of their parent files completely. It might be useful to give the merging program an "exception list" of mods which are meant to be used as pure replacers (might also be overly complicated though).

Good idea, shouldn't be hard to implement.

- Most documentation states that esms can't modify objects from other esms. This is wrong, it's a limitation of the Construction Set which has been misunderstood as a limitation of the engine. For an object merger, it's probably best to just treat esms and esps identically except that esms come first in load order (not sure what your program does).

That's what it currently does (hence the slowness).

Thanks for the above, and for the encouragement.

For people with the time and inclination, and knowledge of the inner workings of esp files (such as yourself), a helpful thing to do would be to run a simple merge of a handful of mods with output directed to an ESTXT file, eg

esper -r -o out.estxt plugin1.esp plugin2.esp plugin3.esp

And then look over that file and see if you can see any glaring errors.

» Mon Mar 14, 2011 10:44 pm

Yup, esper processes recurses through all subrecords for each record, and through all slots (member variables) for each subrecord.

Esper does take this into account. The function "merge-method" chooses the method of record merging that should be used for each record/subrecord combination.
The methods :append-multiple and

verwrite-multiple do this "accumulative" merging you talk about. Whether it works correctly is another matter...

Good idea, shouldn't be hard to implement.

That's what it currently does (hence the slowness).

Thanks for the above, and for the encouragement.

For people with the time and inclination, and knowledge of the inner workings of esp files (such as yourself), a helpful thing to do would be to run a simple merge of a handful of mods with output directed to an ESTXT file, eg

esper -r -o out.estxt plugin1.esp plugin2.esp plugin3.esp

And then look over that file and see if you can see any glaring errors.

Sorry I don't have Morrowind installed right now or else I would test this out. I was wondering though, how does a merge tool (this one or others) handle deleted records, and does it treat deleted records differently if it comes in the middle or if it is the last change?

For example say you have base item A, it is changed in mod 1, then mod 2 deletes it. What ends up in the merge file? Also say you have item B, mod 1 deletes it, and mod 2 changes it, what ends up in the merge file there? (Obviously 2 in these cases is meant simply to represent the newest or last file that modifies said item, it could be the 20th or whatever file that modified said item.)

Overall this tool looks very promising and I'll be sure to use it the next time I reinstall Morrowind!

» Mon Mar 14, 2011 7:08 pm

Sorry I don't have Morrowind installed right now or else I would test this out. I was wondering though, how does a merge tool (this one or others) handle deleted records, and does it treat deleted records differently if it comes in the middle or if it is the last change?

For example say you have base item A, it is changed in mod 1, then mod 2 deletes it. What ends up in the merge file? Also say you have item B, mod 1 deletes it, and mod 2 changes it, what ends up in the merge file there? (Obviously 2 in these cases is meant simply to represent the newest or last file that modifies said item, it could be the 20th or whatever file that modified said item.)

Good question! I actually wasn't aware that plugins, as opposed to save games, could delete records. If someone can point me to an explanation of how plugins handle this I will modify esper.

» Tue Mar 15, 2011 5:19 am

Good question! I actually wasn't aware that plugins, as opposed to save games, could delete records. If someone can point me to an explanation of how plugins handle this I will modify esper.

Not sure so this probably need testing, but I think case A) is not handled on mod loading, and probably could crash the game if/when trying to change the deleted reference at run-time, so maybe the proper merging policy would be: signal conflicting mods, ask for exit or continue; if continue, discard deletion, keep the changed reference.

» Mon Mar 14, 2011 3:14 pm

@eeickythump:

I have still the Enchantment problem because I forgot to make a detailed description:

Armor Integrative Modification modifies Auriel's shield to use a custom enchantment: AIM_Auriels_Shield_Unique_en, in my merged objects.esp esper changes the Enchantment to: AIM_auriels_shie and this is the reason that Morrowind complains about a missing Enchantment. Perhaps a buffer is too small?

» Mon Mar 14, 2011 1:53 pm

I have still the Enchantment problem because I forgot to make a detailed description:

Armor Integrative Modification modifies Auriel's shield to use a custom enchantment: AIM_Auriels_Shield_Unique_en, in my merged objects.esp esper changes the Enchantment to: AIM_auriels_shie and this is the reason that Morrowind complains about a missing Enchantment. Perhaps a buffer is too small?

I think I've just fixed that. I fixed a problem with "ENAM" subrecords for weapons previously, but now I see the same fix was needed for armour and clothing.

Esper beta 8 is now up, with this fix.

Also, in response to some earlier requests, I have uploaded a version which runs as an interactive lisp interpreter -- esper-repl.zip. To use it run "lesper.bat". You can run any of the esper functions at the prompt, or any common lisp function. Command line arguments will not be passed to esper this way; you need to feed them to esper via the MAIN function, eg:

(main "-r plugin1.esp plugin2.esp -o out.esp")

To read the contents of a plugin file into memory (specifically, into a list variable called *records*):

(read-records-from-plugin "plugin.esp")

To write the contents of *records* to a file:

(write-records-to-plugin "out.esp" *records*)

To quit:

(quit)

@abot: Thanks, however I still don't understand -- how can a plugin "delete" (as opposed to alter or not mention) a record?

» Mon Mar 14, 2011 8:34 pm

Awesome! Thanks for the new version.

This was the last big problem with esper. I will try it as soon as possible

» Mon Mar 14, 2011 10:16 pm

@abot: Thanks, however I still don't understand -- how can a plugin "delete" (as opposed to alter or not mention) a record?

Technically, it seems a DELE string identifier followed by a number of 0 bytes is appended right after the object identifier. I have tried deleting "chargen dagger" and Fargoth in 1.esp, and changing their name/stats in 2.esp. It seems loading 1.esp first and 2.esp last is working without problem in-game.
I don't know if this can be considered generally safe, probably a more complete crash test would be of use (trying to modify enchantments, attached sounds etc..., for instance setdelete 1 an object reference still playing an enchantment effect/sound is believed to be a potential crash cause in scripts).
Even if the merger could just discard merging 1.esp and 2.esp in this case. I still think that an advice/log "Hey, mod B is trying to modify something deleted by mod A!" would be useful when revising a mod list.

[EDIT]I don't know if this can be of any help to you, anyway this is the output (out.estxt, NUL skipped) obtained with the command
esper -r -o out.estxt 1.esp 2.esp

(TES3 346 (TES3 HEDR 300 (ESPER::VERSION 1.3) (ESPER::FILE-TYPE :ESP)  (ESPER::COMPANY-NAME "								")  (ESPER::DESCRIPTION "Records merged by Esper Beta 8{{ESPER:NOMERGE}}			  (lotta spaces skipped here)		  ")  (ESPER::NUM-RECORDS 2)) (TES3 MAST 14 (ESPER::STRING-VALUE "Morrowind.esm")) (TES3 DATA 8 (ESPER::INTEGER-VALUE 1234)))(NPC_ 590 (NPC_ AI_W 14 (ESPER::AI-DISTANCE 512) (ESPER::AI-DURATION 5)  (ESPER::AI-TIME-OF-DAY 0) (ESPER::AI-IDLE (60 20 20 20 0 0 0 0))  (ESPER::AI-UNKNOWN1 1)) (NPC_ AIDT 12 (ESPER::AI-HELLO 30) (ESPER::AI-UNKNOWN1 0) (ESPER::AI-FIGHT 30)  (ESPER::AI-FLEE 30) (ESPER::AI-ALARM 0) (ESPER::AI-UNKNOWN2 255)  (ESPER::AI-UNKNOWN3 255) (ESPER::AI-UNKNOWN4 255) (ESPER::AI-FLAGS NIL)) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "misc_com_bottle_12			  ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "torch_infinite_time			 ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 9)  (ESPER::ITEM-NAME "random gold					 ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "common_pants_04				 ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "common_shoes_04				 ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "common_shirt_04				 ")) (NPC_ NPCO 36 (ESPER::ITEM-COUNT 1)  (ESPER::ITEM-NAME "common_ring_04				  ")) (NPC_ FLAG 4 (ESPER::NPC-FLAGS (:NONE? :AUTOCALC))) (NPC_ NPDT 12 (ESPER::NPC-LEVEL 20) (ESPER::NPC-DISPOSITION 50)  (ESPER::NPC-REPUTATION 0) (ESPER::NPC-FACTION-RANK 0)  (ESPER::NPC-UNKNOWN1 255) (ESPER::NPC-UNKNOWN2 255) (ESPER::NPC-UNKNOWN3 255)  (ESPER::NPC-GOLD 0)) (NPC_ SCRI 11 (ESPER::STRING-VALUE "LocalState")) (NPC_ KNAM 23 (ESPER::STRING-VALUE "b_n_wood elf_m_hair_03")) (NPC_ BNAM 23 (ESPER::STRING-VALUE "b_n_wood elf_m_head_02")) (NPC_ ANAM 1 (ESPER::STRING-VALUE "")) (NPC_ CNAM 9 (ESPER::STRING-VALUE "Commoner")) (NPC_ RNAM 9 (ESPER::STRING-VALUE "Wood Elf")) (NPC_ FNAM 26 (ESPER::STRING-VALUE "Fargoth (level set to 20)")) (NPC_ MODL 14 (ESPER::STRING-VALUE "base_anim.nif")) (NPC_ NAME 8 (ESPER::STRING-VALUE "fargoth")) (NPC_ DELE 4 (ESPER::BLOCK-DATA #<RAW "">)))(WEAP 184 (WEAP ITEX 21 (ESPER::STRING-VALUE "w\\Tx_dagger_iron.tga")) (WEAP SCRI 14 (ESPER::STRING-VALUE "CharGenDagger")) (WEAP WPDT 32 (ESPER::WEAPON-WEIGHT 3.0) (ESPER::WEAPON-VALUE 100)  (ESPER::WEAPON-TYPE :SHORT-BLADE-ONE-HAND) (ESPER::WEAPON-HEALTH 400)  (ESPER::WEAPON-SPEED 2.5) (ESPER::WEAPON-REACH 1.0)  (ESPER::WEAPON-ENCHANT-POINTS 20) (ESPER::WEAPON-CHOPMIN 4)  (ESPER::WEAPON-CHOPMAX 5) (ESPER::WEAPON-SLASHMIN 4)  (ESPER::WEAPON-SLASMAX 5) (ESPER::WEAPON-THRUSTMIN 5)  (ESPER::WEAPON-THRUSTMAX 5) (ESPER::WEAPON-FLAGS NIL)) (WEAP FNAM 22 (ESPER::STRING-VALUE "Iron Dagger (changed)")) (WEAP MODL 20 (ESPER::STRING-VALUE "w\\W_iron_dagger.nif")) (WEAP NAME 15 (ESPER::STRING-VALUE "chargen dagger")) (WEAP DELE 4 (ESPER::BLOCK-DATA #<RAW "">)))

[EDIT]skipped a few spaces for better forum post compatibility, replaced "object" with "object reference", corrected chargen_dagger to "chargen dagger"

» Mon Mar 14, 2011 2:19 pm

Technically, it seems a DELE string identifier followed by a number of 0 bytes is appended right after the object identifier. I have tried deleting "chargen dagger" and Fargoth in 1.esp, and changing their name/stats in 2.esp. It seems loading 1.esp first and 2.esp last is working without problem in-game.
I don't know if this can be considered generally safe, probably a more complete crash test would be of use (trying to modify enchantments, attached sounds etc..., for instance setdelete 1 an object reference still playing an enchantment effect/sound is believed to be a potential crash cause in scripts).
Even if the merger could just discard merging 1.esp and 2.esp in this case. I still think that an advice/log "Hey, mod B is trying to modify something deleted by mod A!" would be useful when revising a mod list.

Thanks! It looks like 'deleted' records contain a subrecord named DELE.
Esper as is should process things correctly - when merging several versions of a record, if one of them has a type of subrecord which the other lacks, then a copy of the new subrecord will be added to the "merged" record. So a DELE subrecord in any input record will carry over into the merged record. There is no way that merging can undelete a record.

With this information I have added a new command line option, --nuke. By default, merging records will still merge 'deleted' records correctly but will also remember their deleted status, so they will be written to the output plugin but marked as "deleted records".

If you use --nuke, any records marked as deleted will be DISCARDED when writing the output plugin.

I have uploaded beta 9, containing this change. I have not update the interactive lisp download as this is a very minor change.

» Mon Mar 14, 2011 5:00 pm

I don't know why, but esper ignores all records made by plugins which also get touched by PirateLords Trade enhancements. For example:

LGNPC Pelagiad edits Mebestian Ence (adds a script) and PTE edits him too. (As far as I can tell). In the merged esp all changes made by LGNPC are lost which results in a nice error. (can't find variable: SloganChosen.)

The same with Ice/Bradys Robe Replacer. It changes all robes and PTE edits the base prices: Robe Replacer entries are lost, PTEs prices are still there.

Do you know why this happens?

» Tue Mar 15, 2011 1:38 am

Does the program allow for this? If it doesn't it would be cool if you could make it this easy.

esper -1

esper -r 9, 4, 25, 10, 12, 2 -o mergedobjects.esp //With each number representing a mod based on your load order

Also, this takes the order into account right, and if so is it in ascending or descending order? I think it is extremely important to be able to choose merging order and priority without changing your load order. If it were as easy as just typing the respective number in your load order list I wouldn't need a GUI at all it would be 10x easier than TEStool since it only goes by activated mods and is based on load order which is so annoying. When dealing with complicated mod lists many times you need or prefer to have your objects merged in a different order than what you want them to load as and having to disable all the ones you don't want to merge.... arg so annoying. Also, does this merge references from .esm files.

» Tue Mar 15, 2011 3:13 am

You are wrong. Don't use the load order numbers, use the esp file names. For example:

esper -r "Mod1.esp" "Mod 2.esp" -o merged.esp.

» Mon Mar 14, 2011 5:02 pm

I don't know why, but esper ignores all records made by plugins which also get touched by PirateLords Trade enhancements. For example:

LGNPC Pelagiad edits Mebestian Ence (adds a script) and PTE edits him too. (As far as I can tell). In the merged esp all changes made by LGNPC are lost which results in a nice error. (can't find variable: SloganChosen.)

The same with Ice/Bradys Robe Replacer. It changes all robes and PTE edits the base prices: Robe Replacer entries are lost, PTEs prices are still there.

Do you know why this happens?

What command line are you using? Are you only merging 2 plugins each time?