Land Magic v0.3 by DarklyDreaming

What's New?
v0.3:
-Added "Bashed Patch, 0.esp" to the ignore list so you can put the LandMagicPatch.esp before the bashed patch(in fact now you should), if your bashed patch is named something else, well I'm sorry.
-Fixed the formID conversion, apparently formID starting by more than (number of master)+1 are part of the current mod.
-Cleaned up a bit the program so it wouldn't hang up when handling certain errors(basically changed all return error values to throw exceptions instead).
-Now it doesn't require all the mods to be there, only those it modifies(yay!).
-Desactivated the "Save repairs even if can't repack the Heightmap properly" because of a bug in repacking if you check that option.

v0.2b:
-Quick fix, I had forgotten to load the esm before the esp(doh!).
-It also checks that the masters are loaded before loading a mod, and will stop there if it's not the case(rearrange your load order using OBMM/Wrye Bash if that happens).

v0.2:
-Fixed a huge bug(for those curious I had forgotten to change the formIDs from local to global context before checking for duplicates).

v0.1:
-First release.


What is this tool for?
This tool fixes(or tries to) tears in the landscape.
This usually occurs if mods change the same group of cells but not totally overwrite each others.

Do I need this?
If you noticed tears in landscape you may want to give this tool a try.
Or by curiosity you might want to try it to see if it detects any tears .

How do I Use this tool?
Unpack LandMagic.exe anywhere, launch it and click on the "Do The Magic!" button, yep it's that complicated!

What is the checkbox "Save repairs even if can't repack the Heightmap properly" for?
Well the heightmaps in Oblivion are packed so that each map can only have a difference of -128 to 127 height units to the previous point.
When repacking the tool might notice that it can't repack the cell properly, the difference between 2 points not being in those limits.
By default(checkbox unchecked) the program won't save the cell at all if it detects 1 point that can't be repacked properly.
With this box checked the program will force the maximum value(ie -128 or 127 depending on which limit was reached) and save the cell.

So should I use this option?
Because of a problem I desactivated it for the moment...

Is this tool perfect?
As the version number indicates it is not .
-It doesn't(yet) fixes textures not blending together on cell borders.
-It doesn't compress the fixed LAND records.

When should I rebuild the patch?
Everytime you add/modify a mod(might not be important if there is no landscape in the mod but, are you sure about it? ).

I'm not happy with the result of the patch, how can I desactivate it?
Simply uncheck it from the mod list or delete LandMagicPatch.esp.

Will the source code be released?
It might once I'm satisfied the program is clean enough for public viewing .
If you're looking for esp/esm parsing code in C++ I'd rather advise you the CBash source code which is far more complete:
http://www.tesnexus.com/downloads/file.php?id=30154

Where should I put LandMagicPatch.esp in my load order?
Doesn't really matter as long as it's after its masters.

What is LandMagic_x64.exe?
If you're using a 64 bit OS(either XP 64bit, Vista 64bit or Windows 7 64bit) you can use this version.
It should be slightly faster and gets rid of the 2Gb memory limitation if you have huge lands/mods to check for tears.

Thanks to:
-Waruddar for pointing out the correct way to calculate blocks and subblocks of a cell, also for demonstrating that my "invalid formIDs" were in fact not .
-All those who contributed documentation on the Tes4Mod:Programmers part of the uesp: http://www.uesp.net/wiki/Tes4Mod:Programmers

http://tesnexus.com/downloads/file.php?id=30519

this is absolutely incredible!!!

Glad to see you got this to a working state

[EDIT] What does this mean?

By Zeus! 5815 cells fixed(and 3580 that couldn't get fixed, that's quite a lot).
As for the error that's worrying, Program Exceptions are exceptions set by me and SHOULDN'T happen .

In this case it happens during the deletion of a record or a grup from the tree(just noticed I forgot to change the exception text from record to grup in the Delete_Grup function ). It happens because a reference was passed for deletion and the reference couldn't be found in the list of grups/records.

This should not happen as the reference was found in the list just before it was deleted(or in the case of grup it was specifically linked to the cell and shouldn't have been deleted before the cell)...

Frankly I have no idea why that's happening, it shouldn't happen...

I will try to make a version with more precise exception catching for this part to see where it fails.
Just by curiosity, do you get the error with the 2 different versions(32 and x64?) and what is your list of mods(in particular which mod causes so many cells to be fixed?)?

This program sounds like it could be absolutely amazing!

...If I could get it to work. Get this error.

Loads everything correctly, until this. Then nothing happens after that. Weird.



Any ideas? My game works fine, very stable, and I've never got any errors like that before with anything else...

Here's my load order, if it helps:

EDIT: REMOVED LOAD ORDER TO PREVENT PAGE CLUTTER .

Gah, hate the new forum skin.

@ {frt}Tomato: Most likely the tool is only designed to accommodate 255 ESM/ESP files, since that's the total the game allows to be active at once. But the Bash Patch allows you to merge and disable all those marked in your load order with **. My guess would be that this new tool is trying to load these ESPs too and thus ending up with more than 255 ESM/ESP files, causing it to stumble. Another possibility could have been that you had 255 active ESM/ESP files and the tool creates a 256th ESP, but there are no Form ID ranges left to use. However since your load order stops at F7 this isn't likely.

DarklyDreaming, in layman's terms, how does the tool work? Does it start by loading all LAND records into a single new ESP, taking last loaded as winning record, then look for tears in the resulting ESP?

Ahah that makes total sense. Bummer, means I can't use it. Unless...I temporarily remove everything that I know/hope doesn't edit landscapes until I do have 254 ESPs. Though that's a lot of effort and sense, too much for me . But yeah, if this program can account for that, and does actually work, brilliant. Landscape tears have always annoyed me, and with a load list like mine there's bound to be loads .

EDIT: No, actually, I loaded the load order into word down to the Guards of Cyrodiil ESP, then used word count to find the number of lines. According to word, its the 166th mod in my load order. Maybe an error in the ESP or something?

Well, it was just a possibility Have you tried disabling that one ESP and seeing whether things continue fine? It could well be an issue with just that ESP. The error might be stating that the ESP contains one or more bad Form IDs, rather than complaining that it's run out of new IDs in the range (which is what I assumed the error to mean).

Silly me, seemed to work fine without that ESP.



Also got this, wonder what it means?

Program Exception:Couldn't find the record to delete...

I know know it means the program didn't work. I will continue to investigate this .

do you have any before/after screenshots of Land Magic in action?

The program only loads what is in the load list, not everything.
The error occurs when there is a badly formed ID(ie if the mod has 4 masters and the formID starts with 05 or more), it can't translate the formID to the patch context(with all mods loaded) and fails miserably . I can probably add some exception handling at that point to continue loading and ignore the record, for the next version.

And this "Program Exception:Couldn't find the record to delete..." is annoying me >.<.
I will dig a bit and if I don't find the cause I will upload a version with more detailed output for the error.

Well it loads all mods sequentially as would Oblivion(based on the timestamp), if loading a new mod it detects at the same place in the hierarchy of its "internal tree" that the record already exists then it deletes the old record and replaces it.
For the purpose of the tool it only loads the WRLD Top grup and every grup inside and only the WRLD/CELL/LAND records.

Once it's done it links every WRLD to its associated grup(if there is one), every CELL to its associated grup(if there is one yet again). That's the "Optimizing tree part".

Once everything is loaded and optimized it looks for tears in each worldspace sequentially and in each worldpsace it processes every cell sequentially.

Btw giving this explanation I wonder if I messed up something in the duplicate detection...I think I may check the formID before putting it in the proper patch context! Damn, checking now .

edit:
Ok I messed up, don't use this version, will upload a new one tonight...
Btw this is the reason for the "Program Exception:Couldn't find the record to delete..." as it registered the cells more than once and so attributed one grup to multiple cells...


edit2:
Ok version 0.2 uploaded, if you encountered the "Couldn't find the record to delete" bug then this should fix it .

Version 0.2 works... the "Couldn't fine record to delete" nonsense is gone However, of 4676 records, 2714 of them couldn't be repacked properly... should I be worried about that?

Ahh nice, I was worried for a moment that it was something else(which I couldn't fathom ).
And it's quite a lot of records, I'm not really sure what to make of it.

Can you paste the part where it says how many change it made to each worldspace?
You can use this: http://pastebin.com/ to "upload" the list, just paste the url here once you're done(please don't post the whole thing here as the forum is quite terrible atm).

http://pastebin.com/1ku2Mtsf

Land Magic v0.2 by DarklyDreamingMagic is starting, close your eyes!CModManager initializedCTreeManager initializedNow parsing all the active mods...----Loading Unofficial Oblivion Patch.espLoading Unofficial Shivering Isles Patch.espLoading Enhanced Economy.espLoading DLCShiveringIsles.espLoading Francesco's Optional New Adventurers.espLoading All Natural - Real Lights.espLoading All Natural.espLoading Oblivion.esmLoading All Natural - SI.espLoading Cobl Main.esmLoading Alternative Start by Robert Evrae.espProgram Exception:Error formID, out of range formID!

Well, that was disappointing

Since I didn't get very far with this, is that order the order it's actually loading things in? If it is, then I think you need to adjust your logic to load the ESMs first, and then the ESPs after that.

Soo, here is a fun little question...

I've always dreamt of the day when I could play Elsweyr Anequina and Valenwood Improved DV together, but was stopped cold by the enormous amount of land tears and height map differences. So, is there a good possibility that your handy little gem could fix these issues?

This also may be beneficial for http://www.tesnexus.com/downloads/file.php?id=27235

3942 in Tamriel .
I got 27 here but I see you have all the unique landscapes, that's probably the reason, in fact there may be teared landscape on purpose in those, I'm not sure(I don't use them).




Ah yes, I didn't notice the problem here as my modlist is BOSS-ordered but I should indeed load the esm first...
I should also check that the master of each mod are loaded prior to loading the mod in fact...
Well I will probably do that this week end .

As for the "out of range formID" on Alternative Start by Robert Evrae that's strange because I'm also using this mod and I don't get this error...

@far327:
It might but if the height difference is really huge it will probably fail.

edit:
Taking a break from coding for a few days, DA: Awakening is coming out .

Maybe it showed up as out of range because the load ordering is out of sequence?

Gave this a try. Seamed to Work fine , Loaded everything according to the Load order. I also have some interesting #'s

Checking Worldpsace: Tamriel
Number of cells fixed: 3894

Checking Worldpsace: SEWorld
Number of cells fixed: 35
Checking Worldpsace: DreamWorld
Number of cells fixed: 4
Checking Worldpsace: TGTempleOfTheEmperorZero
Number of cells fixed: 3
Checking Worldpsace: E3Kvatch
Number of cells fixed: 3
Total number of cells fixed: 4484
Number of cells that couldn't get repacked properly: 2642

I just picked out some interesting ones. Did save the log if ya want it.I just wonder why so many for Tamriel ?

A question of order...BASH Patch, LM (Land Magic) Then Run TES4LOD?

Also would this go Before the Patch or after? Would need to Load very late but I do like to keep BASH patch as last.

Edit Forgot this bit...
Now calculating heightmaps...
Number of HeightMaps processed: 70991.<<< Thats a lot...lol and did this Fast too.

No, none of the ULs have land tears on purpose, and none of them are likely to have any by accident. ishmaeltheforsaken is using a number of my UL compatibility patches which do a lot of land tear fixes themselves. This is why I earlier asked how the tool works, as I was curious whether it would take into account later ESPs in the load order intentionally overriding land records specifically to repair land tears and resolve floating/buried objects.

The order the esp are loaded in should be ok, in fact it should be the same order as your load order(well it should be ok as long as your load order isn't messy).
Only thing is I forgot to load the esm first and that can cause some big problems in some cases...
In fact this part is already fixed but I also want to make sure the masters are loaded before each module starts to load and that part is not done yet.

Makes me wonder, what happens if 2 files got each other as masters?
Does Oblivion go into an eternal loop?

edit:
v0.2b is out, should fix load order issues(first bug free version hopefully).
And if it detects a load order problem it will detail it and stop right there.

Land Magic v0.2b by DarklyDreamingMagic is starting, close your eyes!CModManager initializedCTreeManager initializedNow parsing all the active mods...----Loading Oblivion.esmLoading Cobl Main.esmLoading All Natural Base.esmLoading Open Cities Resources.esmLoading CM Partners.esmLoading Francesco's Leveled Creatures-Items Mod.esmLoading Francesco's Optional New Items Add-On.esmLoading Francesco's Optional New Creatures Add-On.esmLoading VASE - core.esmLoading TamrielTravellers.esmLoading Cybiades.esmLoading Children Of Rourken.esmLoading Kvatch Rebuilt.esmLoading TR_OoT_Main.esmLoading Windfall.esmLoading GTAesgaard_2.esmLoading Unofficial Oblivion Patch.espLoading Unofficial Shivering Isles Patch.espLoading Enhanced Economy.espLoading DLCShiveringIsles.espLoading Francesco's Optional New Adventurers.espLoading All Natural - Real Lights.espLoading All Natural.espLoading All Natural - SI.espLoading Alternative Start by Robert Evrae.espProgram Exception:Error formID, out of range formID!

Load order processing is ok now obviously, but it's not having anything to do with that alternate start. It doesn't have any other ESPs as masters, so it shouldn't have any strange formIDs going on. It might be useful if the application specified the formID it thinks is out of range. It's impossible to diagnose without that other than to drop it from the load order, which I can't do without in turn having it turn off my bashed patch.

Ok I uploaded a "debug" version on tesnexus.
It should give what kind of record it was processing when the error occured and the formid and the list of correspondance for the formids for the mod being processed...

Loading Alternative Start by Robert Evrae.espInvalid formID: 203D1B2Listing possible formIDs:Formid starting by 0Corresponds to: Oblivion.esmFormid starting by 1Corresponds to: Alternative Start by Robert Evrae.espCaught exception processing a XCLR entry: 2Program Exception:Error formID, out of range formID!

Nice. Your utility exposed a mistake I made when I was altering my copy to support Leyawiin Reborn. I hadn't noticed that invalid region info had been saved to the cells when I placed the new markers. Your debug info by itself wasn't enough to identify the problem, but TES4Edit's "check for errors" function pinpointed it immediately. So that was my mistake, uncovered by your utility

Loading The Lost Spires.espInvalid formID: 200F8ECListing possible formIDs:Formid starting by 0Corresponds to: Oblivion.esmFormid starting by 1Corresponds to: The Lost Spires.espCaught the exception processing a ATXT subrecord!Program Exception:Error formID, out of range formID!

That one TES4Edit can't find though, and I don't know if this is the result of an error I made or the result of something else because the valid form ID for the actual texture should be 0100F8EC, but it seems something is referencing it as 0200F8EC which is obviously wrong. Might be helpful if when subrecord errors are found, the record they're attached to is displayed too.