[WIP] New merge objects program

Post » Tue Mar 15, 2011 1:31 am

Took it for a quick spin. Couple of things on the text files: for us poor benighted Windows users could you add a .TXT file extension to them and convert to CR+LF from LF? Goes against the grain I know, and john.moonsugar'll probably want to throttle me...

Output:
I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -lESPER version Beta 1Released under GPL; see the file LICENSE for details.; Warning: INI file #P"./Morrowind.ini" does not exist.; While executing: ESPER::ESPER, in process toplevel(2).I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -l -d "I:\Bethesda Softworks\Morrowind.testing"ESPER version Beta 1Released under GPL; see the file LICENSE for details.Bloodmoon.esmMorrowind.esmTribunal.esmuberTesting.espOdai-Cascade V2.5_TREdit.espI:\Bethesda Softworks\Morrowind.testing\Data Files>esper -a -r -o bigmerge.estxtESPER version Beta 1Released under GPL; see the file LICENSE for details.> Error: No such file or directory : #P"/Bethesda Softworks/Morrowind.testing/Data Files/./Morrowind.ini"> While executing: CCL::MAKE-FILE-STREAM, in process toplevel(2).;;;;;; #<PROCESS toplevel(2) [Active] #x8EFA036> requires access to Shared Terminal Input;;; Type (:y 2) to yield contro^CI:\Bethesda Softworks\Morrowind.testing\Data Files>esper -a -r -d "I:\Bethesda Softworks\Morrowind.testing" -o bigmerge.estxtESPER version Beta 1Released under GPL; see the file LICENSE for details.> Error: value #P"I:/Bethesda Softworks/Morrowind.testing/Data Files/Bloodmoon.esm" is not of the expected type SEQUENCE.> While executing: CCL::SEQUENCE-TYPE, in process toplevel(2).;;;;;; #<PROCESS toplevel(2) [Active] #x8EF9F56> requires access to Shared Terminal Input;;; Type (:y 2) to yield contro

User avatar
Sheila Reyes
 
Posts: 3386
Joined: Thu Dec 28, 2006 7:40 am

Post » Mon Mar 14, 2011 9:06 pm

Took it for a quick spin. Couple of things on the text files: for us poor benighted Windows users could you add a .TXT file extension to them and convert to CR+LF from LF? Goes against the grain I know, and john.moonsugar'll probably want to throttle me...
[


Thanks for the report.
I can make it recognise the .TXT extension as an alternative to .ESTXT easily.
I probably won't do the CRLF conversion -- all good programmers' editors (eg Notepad++ for windows) can work with unix line endings seamlessly.

I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -l
ESPER version Beta 1
Released under GPL; see the file LICENSE for details.

; Warning: INI file #P"./Morrowind.ini" does not exist.
; While executing: ESPER::ESPER, in process toplevel(2).

I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -l -d "I:\Bethesda Soft
works\Morrowind.testing"
ESPER version Beta 1
Released under GPL; see the file LICENSE for details.

Bloodmoon.esm
Morrowind.esm
Tribunal.esm
uberTesting.esp
Odai-Cascade V2.5_TREdit.esp


Is this the correct load order for those files? (Is this how they look in Wrye Mash sorted by load order?)

I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -a -r -o bigmerge.estxt

ESPER version Beta 1
Released under GPL; see the file LICENSE for details.

> Error: No such file or directory : #P"/Bethesda Softworks/Morrowind.testing/Da
ta Files/./Morrowind.ini"
> While executing: CCL::MAKE-FILE-STREAM, in process toplevel(2).




The rest looks like simple problems with filename handling - due to inadequate testing on Windows (developed on Linux as you can probably tell).
*Should* be trivial to fix -- will do so tomorrow.
User avatar
Solène We
 
Posts: 3470
Joined: Tue Mar 27, 2007 7:04 am

Post » Tue Mar 15, 2011 1:46 am

Thanks for the report.
I can make it recognise the .TXT extension as an alternative to .ESTXT easily.
Oh it's not recognising file extensions its just in the archive the readme and license are "readme" and "license" and not "readme.txt" and "license.txt" without a file extension Windows frreaks out and doesn't know what program to handle them with

I probably won't do the CRLF conversion -- all good programmers' editors (eg Notepad++ for windows) can work with unix line endings seamlessly.
Oh, no problem for me it's just I don't think everyone round here will have a Notepad replacement that can cope with Unix-style line endings

Is this the correct load order for those files? (Is this how they look in Wrye Mash sorted by load order?)
No. ESPs are correct but it's Morrowind > Tribunal > Bloodmoon. Should've been more explicit.

The rest looks like simple problems with filename handling - due to inadequate testing on Windows (developed on Linux as you can probably tell).
*Should* be trivial to fix -- will do so tomorrow.
Well the problems I see are
(i) Having to explicitly specify the Morrowind.ini directory (default doesn't work, I guess "./" isn't interpreted correctly on Windows as meaning the parent directory of the current working directory).
(ii) The "not of the expected type SEQUENCE." error for Bloodmoon.esm, perhaps related to the load order listing.
User avatar
Krystal Wilson
 
Posts: 3450
Joined: Wed Jan 17, 2007 9:40 am

Post » Tue Mar 15, 2011 2:30 am

I tried giving this a shot as I'm getting a new character together and needed a merged obj's for her. Unfortunately after unpacking the exe and clicking on it, nothing happens. It flashes on screen briefly and shuts down before I get a chance to do anything with it.

Installed to Data Files, on a XP machine, the install instructions seemed simple enough though the 'usage' instructions seemed rather unclear, I didn't manage to get far enough to have problems with those however.
User avatar
clelia vega
 
Posts: 3433
Joined: Wed Mar 21, 2007 6:04 pm

Post » Mon Mar 14, 2011 6:36 pm

I tried giving this a shot as I'm getting a new character together and needed a merged obj's for her. Unfortunately after unpacking the exe and clicking on it, nothing happens. It flashes on screen briefly and shuts down before I get a chance to do anything with it.

Installed to Data Files, on a XP machine, the install instructions seemed simple enough though the 'usage' instructions seemed rather unclear, I didn't manage to get far enough to have problems with those however.
It's a command line utility so you have to get a DOS prompt up (Start > Run... then type CMD) and navigate to your Morrowind install (e.g. CD C:\Program Files\Bethesda Softworks\Morrowind\Data Files) and then run it with the command line arguments in the TXT file
User avatar
Big Homie
 
Posts: 3479
Joined: Sun Sep 16, 2007 3:31 pm

Post » Mon Mar 14, 2011 10:24 pm

It's a command line utility so you have to get a DOS prompt up (Start > Run... then type CMD) and navigate to your Morrowind install (e.g. CD C:\Program Files\Bethesda Softworks\Morrowind\Data Files) and then run it with the command line arguments in the TXT file



Is that in the readme? Bit complex for the humble masses, isn't it? Ok, I'll try that, thanks for the info!
User avatar
Jeremy Kenney
 
Posts: 3293
Joined: Sun Aug 05, 2007 5:36 pm

Post » Mon Mar 14, 2011 4:31 pm

Is that in the readme? Bit complex for the humble masses, isn't it? Ok, I'll try that, thanks for the info!
Nope, what I was thinking of was suggesting a batch file or a set of batch files that could be included with the program. People'd just need to double click on them to get this to work
User avatar
Michelle Smith
 
Posts: 3417
Joined: Wed Nov 15, 2006 2:03 am

Post » Mon Mar 14, 2011 9:17 pm

I think some kind of UI is definitely needed. I've got well over a hundred mods I need to enter in separately to tell it to merge objects? At that point it would be easier to put up with TesTool's 10 at a time and merge them all into one with the CS when I'm done.

Oh and in case any other clueless people come by wanting to try this out you have to type "chdir" into the command line utility in order to change the directory to your morrowind\data folder (yes you must type the whole thing out correctly first) and the commands Dragon mentioned are best taken as from the 'example' portion of the readme, yes there are spaces in the commands and the readme was best opened (for me at least) without unpacking the file. As WinRar will read it fine but XP won't.
User avatar
Kelly John
 
Posts: 3413
Joined: Tue Jun 13, 2006 6:40 am

Post » Mon Mar 14, 2011 3:21 pm

Hi guys.
Beta 2 is uploading now.
To use it, put it in your "Data Files" directory and always include "-d .." in the command line.
For example:

esper -p plugin1.esp plugin2.esp -d ..

I am still having some problems with filenames/directory names. One problem is with handling of filenames that contain apostrophes ('). Such as Beryl's Head Replacer.esm. For now it cannot find those files, so if you desperately want to try esper on such files you need to rename them (the files themselves, and their entries in the ini file).

Thanks for trying it.
esper -p *some plugins* -d .. definitely works ie generates an output plugin.

And yes, as the docs clearly state, it's a command line utility. You need to install the esper exe file in the Data Files dir, then open "command prompt", then type:
cd "\games\Morrowind\Data Files"esper --help


substitute your MW directory in the above, obviously.
User avatar
Strawberry
 
Posts: 3446
Joined: Thu Jul 05, 2007 11:08 am

Post » Mon Mar 14, 2011 7:00 pm

Also: I will soon add an option where it will read the names of plugins from a text file, one per line.
User avatar
Melly Angelic
 
Posts: 3461
Joined: Wed Aug 15, 2007 7:58 am

Post » Tue Mar 15, 2011 3:35 am

@DesertRat: would this involve merging CELL records or LAND records? The structure of LAND records seems to be a mystery in the MW modding community (please correct me if wrong). Probably involves heightmaps and such. The data in CELL records includes such things as whether the cell is interior or exterior, lighting, location of exists etc.

Let me know if you need any info on the TES3.LAND record. It isn't quite as simple as most people first expect.
User avatar
Alan Cutler
 
Posts: 3163
Joined: Sun Jul 01, 2007 9:59 am

Post » Tue Mar 15, 2011 4:03 am

Let me know if you need any info on the TES3.LAND record. It isn't quite as simple as most people first expect.


I would appreciate any info on LAND records. Is enough known about their structure to do anything useful with them?
User avatar
Add Meeh
 
Posts: 3326
Joined: Sat Jan 06, 2007 8:09 am

Post » Mon Mar 14, 2011 3:44 pm

I would appreciate any info on LAND records. Is enough known about their structure to do anything useful with them?


If you want more info, the two things to check out are Wrye Mash (allows updating the world map, so may have something on the TES3.LAND.WNAM record) and MGE which should have everything I have written here plus code about the TES3.LAND.VCLR record.

TES3.LAND	INTV (Position)	VHGT (Height data)	VNML (Normal data)	VTEX (Texture data)	WNAM (9*9 chars, possibly map data. IDK)	VCLR (65*65 chars). At a guess vertex colours.


Holds the grid position of the cell. 8 bytes.
struct INTV {			int x;			int y;};


Holds the height data offsets (see below for details)
struct VHGT {			float heightOffset;			char  heightData[65*65]; //verts			short unknown1;			char unknown2;		};


Normals for every vertex
struct VNML {	struct NORMAL {		 char x;		 char y;		 char z;	 };	NORMAL normals[65*65];};


Holds texture indexes. These releate to the textures as defined in the current esp. These differ across esp. The index refers to TES3.LTEX.INTV.
IIRC the textures are not in a 16x16 grid, but in a 4x4 grid in a 4x4 grid.
struct VTEX {	short index[16*16];};


Code for converting TES3.LAND.VHGT into a standard heightmap (If you can write Lisp, I am sure you can understand C++).
   	const int LAND_VERT_WIDTH = 65;	float offset = vhgt->heightOffset;	for (int y = 0; y < LAND_VERT_WIDTH; y++) {		offset += vhgt->heightData[y*LAND_VERT_WIDTH];		//the vertex at [0][y] has a height of (float)offset*8		float pos = offset;		for (int x = 1; x < LAND_VERT_WIDTH; x++) {		   pos += vhgt->heightData[y*LAND_VERT_WIDTH+x]; 		   //the vertex at [x][y] has a height of  pos*8;		}	}

User avatar
Cash n Class
 
Posts: 3430
Joined: Wed Jun 28, 2006 10:01 am

Post » Tue Mar 15, 2011 3:35 am

If you want more info, the two things to check out are Wrye Mash (allows updating the world map, so may have something on the TES3.LAND.WNAM record) and MGE which should have everything I have written here plus code about the TES3.LAND.VCLR record.




Thanks, that's awesome!
User avatar
Grace Francis
 
Posts: 3431
Joined: Wed Jul 19, 2006 2:51 pm

Post » Tue Mar 15, 2011 1:37 am

With Beta 2
I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -l -d "I:\Bethesda Softworks\Morrowind.testing"ESPER version Beta 1Released under GPL; see the file LICENSE for details.Command line arguments:("-l" "-d" "I:\\Bethesda Softworks\\Morrowind.testing"); Warning: Morrowind.ini not found in directory NIL.; While executing: ESPER::ESPER, in process toplevel(2).I:\Bethesda Softworks\Morrowind.testing\Data Files>esper -l -d I:\Bethesda Softworks\Morrowind.testingESPER version Beta 1Released under GPL; see the file LICENSE for details.Command line arguments:("-l" "-d" "I:\\Bethesda" "Softworks\\Morrowind.testing"); Warning: Morrowind.ini not found in directory NIL.; While executing: ESPER::ESPER, in process toplevel(2).
Didn't think the last command would work. Note the list-active plugins worked fine with beta 1

Should only be one backslash in the path (which you probably already knew).

Not sure if this would help with finding morrowind.ini but if I'm in the Bethesda Softworks\Morrowind.testing\Data Files directory and I type ..\morrowind.ini then the INI file will open
User avatar
ruCkii
 
Posts: 3360
Joined: Mon Mar 26, 2007 9:08 pm

Post » Mon Mar 14, 2011 1:28 pm

Hi, beta 3 is uploading. Several changes and fixes -- see post #1 in thread.
User avatar
Jeff Turner
 
Posts: 3458
Joined: Tue Sep 04, 2007 5:35 pm

Post » Mon Mar 14, 2011 7:45 pm

Download link in first post is:
http://bitbucket.org/eeeickythump/esper/downloads/esper1.zip

Should be:
http://bitbucket.org/eeeickythump/esper/downloads/esper3.zip

Edit:
OK, this is looking good :thumbsup:
Comparing old and current records: "WEAP" "steel claymore of hewing"Merged records: #   #   #   #   #   #> + #   #   #   #   #   #> -> #   #   #   #   #   #>Storing new record: "WEAP" "boethiah's walking stick"Storing new record: "WEAP" "bound_longsword"Storing new record: "WEAP" "bound_mace"Storing new record: "WEAP" "bound_battle_axe"Storing new record: "WEAP" "bound_spear"Storing new record: "WEAP" "bound_longbow"Storing new record: "WEAP" "fiend battle axe"Comparing old and current records: "ENCH" "wizard's staff_en"Merged records: #   #   #   #> + #   #   #   #> -> #   #   #   #>Storing new record: "WEAP" "sunder"Comparing old and current records: "WEAP" "ebony_bow_auriel"Merged records: #   #   #   #   #> + #   #   #   #   #   #> -> #   #   #   #   #   #>Comparing old and current records: "ARMO" "ebony_shield_auriel"#Exiting...
Finally, thought I'd try the Check Resources:
I:\Bethesda Softworks\Morrowind\Data Files>esper --check-resourcesEsper, version Beta 1Released under GPL; see the file LICENSE for details.Command line arguments:("--check-resources")#<SIMPLE-ERROR #x8F2CA36>Exiting...
The List active plugins works fine on this install, btw.
User avatar
Latino HeaT
 
Posts: 3402
Joined: Thu Nov 08, 2007 6:21 pm

Post » Mon Mar 14, 2011 2:49 pm

Definitely keeping my eye on this. :) The only reason I don't merge objects is because of the problems I hear TEStool has with a lot of mods and it didn't like Morrowind Cities/bandits Unlocked.
User avatar
Naomi Lastname
 
Posts: 3390
Joined: Mon Sep 25, 2006 9:21 am

Post » Tue Mar 15, 2011 3:18 am

Edit:
OK, this is looking good :thumbsup:

Excellent!
Comparing old and current records: "WEAP" "steel claymore of hewing"Merged records: #   #   #   #   #   #> + #   #   #   #   #   #> -> #   #   #   #   #   #>Storing new record: "WEAP" "boethiah's walking stick"Storing new record: "WEAP" "bound_longsword"Storing new record: "WEAP" "bound_mace"Storing new record: "WEAP" "bound_battle_axe"Storing new record: "WEAP" "bound_spear"Storing new record: "WEAP" "bound_longbow"Storing new record: "WEAP" "fiend battle axe"Comparing old and current records: "ENCH" "wizard's staff_en"Merged records: #   #   #   #> + #   #   #   #> -> #   #   #   #>Storing new record: "WEAP" "sunder"Comparing old and current records: "WEAP" "ebony_bow_auriel"Merged records: #   #   #   #   #> + #   #   #   #   #   #> -> #   #   #   #   #   #>Comparing old and current records: "ARMO" "ebony_shield_auriel"#Exiting...
Finally, thought I'd try the Check Resources:
I:\Bethesda Softworks\Morrowind\Data Files>esper --check-resourcesEsper, version Beta 1Released under GPL; see the file LICENSE for details.Command line arguments:("--check-resources")#<SIMPLE-ERROR #x8F2CA36>Exiting...
The List active plugins works fine on this install, btw.

Those errors have hopefully been fixed - I did a lot of testing with my 300+ mods today.
The other problem was that error information was not printing properly -- also fixed.

Beta 4 is uploading now -- it incorporates all the changes just mentioned.

Thanks for testing the program - very helpful and motivating.
User avatar
Beast Attire
 
Posts: 3456
Joined: Tue Oct 09, 2007 5:33 am

Post » Tue Mar 15, 2011 3:31 am

When

Merge RECORDS (objects, NPCS, etc) from plugin1 and plugin2 into merged-objects.esp:

esper -r plugin1.esp plugin2.esp -o merged-objects.esp


if these two esp both changed one data, such as one item's value. plugin1.esp changed it to 10, plugin2.esp changed it to 20.
which data will be in the finall merged-objects.esp ? 10 or 20 ?
User avatar
hannaH
 
Posts: 3513
Joined: Tue Aug 15, 2006 4:50 am

Post » Tue Mar 15, 2011 3:26 am

esper -r plugin1.esp plugin2.esp -o merged-objects.esp

if these two esp both changed one data, such as one item's value. plugin1.esp changed it to 10, plugin2.esp changed it to 20.
which data will be in the finall merged-objects.esp ? 10 or 20 ?


Files are processed in the order they are given on the command line. So plugin2 would override plugin1.
User avatar
kirsty joanne hines
 
Posts: 3361
Joined: Fri Aug 18, 2006 10:06 am

Post » Mon Mar 14, 2011 2:02 pm


Beta 4 is uploading now -- it incorporates all the changes just mentioned.
"esper4.zip" is identical to "esper3.zip", checked with a diff progam and it doesn't recognise "--debug"

Thanks for testing the program - very helpful and motivating.
No problem, glad I could help out in a small way!
User avatar
Silvia Gil
 
Posts: 3433
Joined: Mon Nov 20, 2006 9:31 pm

Post » Mon Mar 14, 2011 3:58 pm

"esper4.zip" is identical to "esper3.zip", checked with a diff progam and it doesn't recognise "--debug"


Aargh!
I have just replaced it -- name is still esper4.zip but should be new version.
User avatar
Bigze Stacks
 
Posts: 3309
Joined: Sun May 20, 2007 5:07 pm

Post » Mon Mar 14, 2011 8:13 pm

Not quite sure how this all works but how would this new merger affect PTE? In the readme it says this "If you use Better Clothes, then load the patch file (included) after merged objects. This will correct the errors caused by merged objects process.

Also edit the merged objects file, and remove all the NPC entries, or if you want to be totally precise (but more long winded), the NPC entries that PTE changes.
Leaving the NPCs in has been known to cause crashes with PTE due to their adjusted inventories and negative items."
User avatar
Bek Rideout
 
Posts: 3401
Joined: Fri Mar 02, 2007 7:00 pm

Post » Mon Mar 14, 2011 10:05 pm

Aargh!
I have just replaced it -- name is still esper4.zip but should be new version.
Still appears to be the old version (the EXE has a last modified date of 13th January), doesn't accept "--debug"
User avatar
Brittany Abner
 
Posts: 3401
Joined: Wed Oct 24, 2007 10:48 pm

PreviousNext

Return to III - Morrowind

cron