FallenWizard:
From what i can see, the plugin is the one at fault. Hopefully i can get it to detect this later and resolve it.
This is from topic 'berserkers' in Less Generic Bloodmoon.
2538 points back to 1396

1396224021218415248 <- 2538727742458621392

So. it goes something like this..

1396224021218415248 -> 2538727742458621392 -> 8888360595565054 ->
2698625811958523045 -> 22370229933171529332 -> 16252191152742210872 ->
292198882173514265 -> 31196115712516019343

In the original Bloodmoon.esm
31196115712516019343 -> 1396224021218415248

Easily an infinite loop when it tires to backtrack and see if it's connected or not, since the order got swapped. I've
seen this before, and things like this make my job harder to try and find a solution. I have a few ideas in mind,
but it may take a while. These dialog issues are what makes it take so long to get some of this fixed and working.

I hope you will be able to fix this infinity loop. I can't play Morrowind right now just because there is no proper merging tool available.

Another bug:
--no_dialog_merge doesn't work, at least not with --reference. It still tries to merge the dialogs. If I remove all plugins with infinity loops from my input file and open the merged .esp with Enchanted Editor, I see a lot of INFO and DIAL records.

the --no_dialog_merge just doesn't try injecting/attaching them, and re-sorting them, but didn't prevent them from being added.
However since we don't want them period, I'll modify it to skip them entirely.

Thanks. This is a great step to a working Merge Objects functionality.

I don't think so.

It's okay if you don't care about dialog/quests but bad implementation to require that if you ask me. I want it to work correctly, rather than have an option to disable merging certain types. I've seen first hand a dozen times how badly merged or sorted data can mess up the game. Either it doesn't reference it, or it doesn't know it exists, or it complains, ect.

Good news, Dialog issues are getting better, fewer issues. However i can't seem to get the information i need to resolve the last one. the 'LGNPC_PaxRedoran_v1_20.esp' is still giving me a hard time in this set, and only in one spot (But the rest merged great so good news FallenWizard). Since the dialog so big (B_Nerevarine) I'll just resume that another time when i find a smaller set. It's likely something very simple, but I'm not about to pour over another 200+ entries to figure this out.

With that out, and some relief in sight, I'll hopefully have the next version released by Friday. Mostly bug fixes so no new features just yet.

A note for later, I've added an auto-check after the topic in the current file has finished, so it will tell you if it merged badly, although it may be confusing so I'll print an example. (From PaxRedoran no less)

(-1) 9352135063201312047 <- 20322912486487339 -> 21553316292731128460
(1) 29654270472852226657 <- 4712258911800323314 -> 20322912486487339

(0) - Fine
(1-2) - Too few INFO's linking back
(-1 or lower) Too many linking.

I've put in red the likely culprit, although it may be a different one. The middle one is the INFO in question. The one to the left is the previous (Above), and the one to the right is after. When the program does it's check, it sees if it has a left and/or right side; If it does it adds 1 for each (so with both it starts at 2). When a INFO links to it, it subtracts 1, so it should be 0 when all is well. If it's 1, then it isn't getting enough back, and if it's -1, it's getting too many. They seem to happen in pairs and are usually directly related. If you see this, and it still writes correctly, you can likely just modify the record that's wrong manually, or maybe leave it alone and the game may work (and pester you a bit).

A thought just occurred to me. If i don't worry about getting the specific orders just let it combine flaws; then resort them before relinking them.

Example:
1) Specific person with choice (choice == 3, or something similar)
2) Specific person & some requirement
3) Specific person & disposition
4) Specific person
5) Cell
6) Anything that's left.

If i follow something like that, then my more complex algorithm can just go out the window. If anyone can think of anything else for the specific order(s), as well as things to look for when re-ordering. Re-linking can easily handle itself. It would require a full re-write, but it would likely be the final re-write. But that's for another version.

I am afraid I cannot help for this, but your progress sound amazing and I am eagerly waiting for the next release.

Thanks and keep on the good work! :thumbsup:

Thanks, I'm always glad to get praise and feedback

It's alright if you can't help, just thinking if someone pours through the dialog a lot, they might make note of what should be in what order. Fighting and re-writing the dialog logic so many times, it just makes you want to tear your hair out. If this latest idea is correct, then my current complex logic (and somewhat faulty with some certain combinations of plugins) can just be a distant memory. But that will likely be a few releases later; Unless it becomes a higher priority.

Although if i do it that way, combining plugins that refer to other conversations (outside of the plugins) may not combine right, unless referenced. Hmmm i wonder how i would resolve that, i got a couple ideas. Oh well, progress shall be made.

Good news indeed! Thanks for your hard work to improve the tool. Smartmerger gets more promising every new version

Version:1.4

No new features, but --load_ini, --no_dialog_merge, and --hard_limit have all been fixed, and should work as expected.

The leveled creatures/items list now merges with lowest levels on top, which i believe is correct.

That's about it there. Unless there's outstanding dialog issues, I'm not touching it for a while. I'll consider doing a re-write on the dialog another time with my new idea/theory, which may or may not complain if not referencing the original esm/esp's. But Warnings doesn't directly interfere with gameplay.

bug report: --no_dialog_merge ONLY merges DIAL/INFO records, nothing else. Strange that it merged LGNPC without freezing.

So i see.. I see where the problem is too (tiny comparator!!). Well, for now just don't use that. When you combine plugins, is they don't combine dialog right, and you remove those plugins you should be able to get most of them combined correctly.

And no, although it's only processing the Dialogs, it doesn't merge them. The only thing it is doing for dialog, is copying new topics, which doesn't cause problems anyways. I'll add this fix for the next release.

Possible bug report:

Merging Morrowind.esm, Tribunal.esm, Bloodmoon.esm, Morrowind Patch v1.6.5-BETA.esm, and LGNPC_NoLore_v0_83.esp. SmartMerger hangs while dealing with LGNPC_NoLore, looks like it's happening with DIAL_B_Bivale Teneran. (I've also used BTB's "dirty patch" for MPP 1.6.5, found http://btb2.free.fr/morrowind.html, and get the same hanging. You can also find a link to download MPP 1.6.5 at his site.)

A thought:

I've tried --load_ini again. This time, of course, it's a valid option! However, while it allows you to easily get a list of all your plugins, the Morrowind.ini file organizes the plugins in alphabetical order, not load order. This makes Morrowind.ini pretty much worthless for those of us who have mods for which order matters (even when merging subrecords, still sometimes you want to control which one overwrites). Ideally, Smart Merger would handle the names from the .ini in date order/load order rather than simply reading them, but this is an additional level of complication you might not want to deal with.

For those that use mlox, there's an excellent alternative. It provides a list of plugins in load order: the file is called current_loadorder.out and --input uses it perfectly.

Yes I've seen the same thing on mine. Mine doesn't hang, mostly because the dialog isn't modified a second time. I'm missing something simple before i can fix that. I'm planning on rewriting it using my newer idea. In the next release I'm going to have a 'breakout' counter, where it will not get stuck in an infinite loop anymore (but may be slow in specific dialogs), but won't necessarily have the order right. Guess it's better than stalling completely. Another option, upon finding a situation where it does this problem and recognizes it, it can force re-linking. This is assuming i can't find the problem and resolve it in the next release.

The current implementation for --load_ini should do the load order by number, IE: GameFile0, Gamefile1, Gamefile2 ... Gamefile22 Ect. .If this is wrong, i'll need to see what has to change for that. I'd rather not have to add more complexity and request information from the filesystem. But if it's needed i will.

A note on it eating 100% of the cpu when it hangs, start it with a low priority. 'start /low /b SmartMerger.exe' and i think all following files/options go after. You can put all this in a batch file with any common flags you intend to use as well, and call that instead. If i can find a easy way to get it to request a lower priority from the OS, i'll go that way.

It's not a problem with Smart Merger; it's a problem with the way Morrowind.ini is organized -- namely, alphabetically instead of in load order. Smart Merger correctly reads the order that's there -- but the order in there is wrong.



I appreciate the tip! Fortunately I haven't had issues with it eating too much CPU; I can stop it easily with ctrl-C.

I'm glad to know you already know about this problem and are thinking of ways to get around it.

So i'll have to get the list, and then get the dates to see what order they should be in. Great... I was hoping to avoid going to dates for the specific load order. Looks like for this part i may end up building the GUI using AHK. I don't look forward to this part.

I have tried to merge several grass mod together (.esp size from 5Mo to 30Mo, the result should be a 70Mo .esp), but smartmerger is freezing during the process (at least I waited for a very long time and it never finished). I've also tried merging mods 1 by 1, but did not succeed.

It may be not fully an issue with smartmerger: loading such .esp in the CS is also very slow and use a large amount of memory. But I finally could succeed with the CS, and the final .esp is large but loading fast enough.

If this isn't something that you want to implement I think you could drop the functionality altogether. As I mentioned, mlox produces a file called current_loadorder.out which needs no special handling on the part of Smart Merger (--input reads it perfectly well, since it is just a list of names, one per line). This file lists all the mods currently turned on, in load order, and thus perfectly fulfills the purpose for which --load_ini was initially requested.

I think the only circumstance under which you really need to make --load_ini work is if you want to have that functionality (merge all current mods, in load order) AND you don't want to suggest mlox to people. I think mlox is a very useful tool that anyone serious about managing lots of mods should be using anyway, so I don't think it's a big deal to suggest it. But I'm not you, of course; you may not think this is a good solution.

Well, rather than rely on another tool, i'll see if i can get a simple order/date function done. If i can, then the feature can work correctly, although i think relying on the dates is a bad idea; Mostly since i tend to be working on a moving target some of the plugins i work with (and update) may change over time, changing the load order on you when you/they update.

Imagine 2 mods change the exact same thing (for instance, mod 1 changes NPC X speed to 50, mod 2 changes NPC X speed to 60).
In this case, you could avoid considering loading order planning a merged speed of 55.
But what about positioning coordinates?
And what about text changes (for instance, mod 1 changes NPC name to "John" , mod 2 changes NPC name to "Jack")?
And I'm sure there are plenty of other potential conflicts where one have to win.
It seems to me you need a ordering rule, why don't use the standard (mod timestamp)?
[EDIT]typos

Doing an average or anything like that is likely a bad idea. There are mods that FIX weight, armor values, weapon damage, Ect. I think there's a couple that lower the price on over-priced items. So assume a newbie writes a new dagger, that does 4-10 damage and health of 1000, and priced it at 10000. Another mod says the price was too high and fixes it at 250 and bumps the health down to about 300, do we go for the average of 5102, and 650?

Text changes are just that, changes. I am not able to merge raw text in any way. The mod makers should be doing a well enough job with their updates on text to not worry about that. And usually updates tend to be minor fixes or re-writing the same thing from what i see, to better flavor the plug-in. If the second plug-in changes his name to 'Jack' then i hope he updated all references to jack's name (if any).

I'm hesitant to use the timestamp(s), however for the --load_ini i may just have to. There are cases where merging one particular mod is better after another, even if it has a earlier timestamp. I don't want to worry about updating the timestamp(s) to just pass the other one so it doesn't go out of order, assuming the order even matters for that plugin. Besides, not only would it break the mods and game i compile for myself, but the implementation becomes more complex and can't be controlled, likely not even with --input.

And don't forget, the stronger part of the program for merging, doesn't replace or do whole records, instead it works by subrecords, and if those have multiple parts, sub-parts too. Example WPDT is the weapon Data, and has about 14 fields. Other mergers can't handle that, even if they could handle separate subrecords. That's why order becomes a bit less important, except for certain cases.

Looking into this more now, i see the problem, thankfully the subject is small and easy to follow. It's directly related to swapped INFO's. It seems that it does a match to see if it's connected (present in the previous or future), and succeeds. Since it thinks all's well, it doesn't blank the last record (which could point back before the info), so it keeps the previous NNAM, making yet another infinite loop. Hopefully this will be much easier to fix than all the others.

Note:
Regarding the order of the INFO's, should i make a new sorting routine (and then re-linking) rather than the current injection method, here's what i have so far. If you see anything else, please commentt. These are highest to lowest order. Just a rough draft.

Fixed Journal entry check.
NPC + Journal (Highest first) + Choice
NPC + Journal (Highest first)
NPC + disp + Journal (Highest first) + Choice
NPC + disp + Journal (Highest first)
NPC + disp
NPC
Faction + Rank (highest first) + Dead NPC check (Or more requirements)
Faction + Rank (highest first)
Race
Class
Cell

I agree with Abot that time stamp should be the default 'tie breaker' as that's what the game uses and so it is what most people would expect (if I want mod x to override changes mod y is making I just make sure mod x is newer than mod y, and if that is not the default of this tool then the expected result may not occur). As it stands though, it is easy enough to ensure that time stamp is what is used via the mlox trick listed above.

As far as the INFO ordering question goes, I don't fully understand it all, but I think we are shooting for a most specific to least specific filtering order, so I think it's (based on what you've laid out):
NPC + Journal (Highest first) + Choice + disp
NPC + Journal (Highest first) + Choice
NPC + Journal (Highest first) + disp
NPC + Journal (Highest first)
Fixed Journal entry check.
NPC + disp
NPC
Faction + Rank (highest first) + Dead NPC check (Or more requirements)
Faction + Rank (highest first)
Race
Class
Cell

But then I could very easily be wrong, as I haven't read much on the subject.

Well, i think i have a fix going, and when i say fix, i mean i was able to merge ALL of FallenWizard's mods into a single one. Course a note to remember, if you don't merge the masters and you reference, the duplicate checking should be disabled, otherwise the duplicate ID's in the original esm may not attach right from the plugin, since they were given new ID's. I can't confirm if everything worked perfectly since the masters some of the plugins rely on are missing. But it's very promising

I will post when i have run several more tests, and re-compiled my own list and see how it does. I'll likely be switching to BTB's list of plugins & enhancements, since he seems to be doing what i originally set out to do with my merger and large chain of plugins