How do you properly convert an .esp to an .esm?

Post » Sat Nov 17, 2012 9:06 pm

Has WryeBash been confirmed as having the same issues as TESVSnip? TESVSnip unpacks and repacks all the records which is the source of the problem. WryeBash, on the other hand, just writes out the file to foo.esm with the master flag flipped, or so I've been assuming.

But I've not looked at the source, so I could be wrong. Anyone know for sure?
Wrye Bash only flips the bit and edits that one spot in the file. Unlike Snip which has to write the whole thing back out and loses the data it truncated in the process.
User avatar
Sandeep Khatkar
 
Posts: 3364
Joined: Wed Jul 18, 2007 11:02 am

Post » Sat Nov 17, 2012 1:20 pm

Wrye Bash only flips the bit and edits that one spot in the file. Unlike Snip which has to write the whole thing back out and loses the data it truncated in the process.

I can use wrye bash to convert it to an .esm and I can see the mod in-game and everything but what is currently broken is the dialogue.

I tried converting my mod to .esm via wrye bash and then creating a new NPC with new Dialogue and merge the changes to the .esm via Version Control and it didn't work.
To confirm that the .esm that wrye bash converted is not working: I tried creating an NPC with some simple dialogue but I didn't merge the changes with the .esm. Tried it in-game and it worked out fine.


So what could possibly fix this from what I know of is:

-Converting my mod to .esm successfully by merging the mod with an empty .esm file via Version Control.
I've been trying for hours for 2-3 days now and I still cannot get this working.

-Somehow splitting up the mod into 1 esp and 1esm
The ESP containing all vanilla skyrim edits and the esm containing everything else. I don't even know if "vanilla skyrim edits" is whats breaking the mod.

-Some other conversion method via a program like Wrye Bash
User avatar
SaVino GοΜ
 
Posts: 3360
Joined: Mon Sep 17, 2007 8:00 pm

Post » Sat Nov 17, 2012 12:37 pm

Wrye Bash only flips the bit, keep that in mind. It doesn't generate the necessary ONAM records that version control would build for it. It might be worthwhile to post a request in the Bash thread for that support. I don't think ONAM records should be particularly difficult to make since they're basically just form IDs to indicate edited vanilla items.

When I get around to setting up my USKP-only install I'll be wanting to get VC working so I'll be digging through whatever info exists on the subject. There's a few tutorials scattered about for it.
User avatar
Laura-Lee Gerwing
 
Posts: 3363
Joined: Fri Jan 12, 2007 12:46 am

Post » Sat Nov 17, 2012 8:38 am

Is it possible to convert my mod to esm via wrye bash and then create another .esm that modifies the original esm?

So it would be:
MyMod.esm - Dependent on Skyrim.esm
NewMasterFile.esm - Dependent on Skyrim.esm & MyMod.esm

And then all the future changes go to that new master file.

Does that make any sense?
User avatar
Shannon Marie Jones
 
Posts: 3391
Joined: Sun Nov 12, 2006 3:19 pm

Post » Sat Nov 17, 2012 7:51 pm

Is it possible to convert my mod to esm via wrye bash and then create another .esm that modifies the original esm?

So it would be:
MyMod.esm - Dependent on Skyrim.esm
NewMasterFile.esm - Dependent on Skyrim.esm & MyMod.esm
That'll work. I'd imagine Dawnguard will have both Skyrim.ESM and Update.ESM as a master and its ONAM list would list any overridden cell children in either. Incidentally, ESPs can have other ESPs as masters just like ESMs and the overrides will work as intended (engine doesn't mind, but CK will not allow ESP masters).
User avatar
Sarah Evason
 
Posts: 3507
Joined: Mon Nov 13, 2006 10:47 pm

Post » Sat Nov 17, 2012 8:12 pm

I'm expecting it to be allowed post-Dawnguard, judging from the DLC comment in the CK error box.

I hope so. I'd like to try and set up some abstract interfaces using master files, but it's pretty much impossible without getting multiple master files.
User avatar
Nicole M
 
Posts: 3501
Joined: Thu Jun 15, 2006 6:31 am

Post » Sat Nov 17, 2012 5:55 pm

Alright I will try that. Thanks!
User avatar
KiiSsez jdgaf Benzler
 
Posts: 3546
Joined: Fri Mar 16, 2007 7:10 am

Post » Sat Nov 17, 2012 8:29 pm

Yeah you guys were right. It gives me this error when I try to check in:


---------------------------
No
---------------------------
Sorry, until we're working on DLC the correct answer is No. You may need to remove an unwanted master file dependency from your plugin.
---------------------------
OK
---------------------------


Is there a way to get around this?

EDIT: Would it help to add MyMod.esm as a master file dependency via Wrye Bash?
User avatar
Naazhe Perezz
 
Posts: 3393
Joined: Sat Aug 19, 2006 6:14 am

Post » Sat Nov 17, 2012 6:27 am

If you still can't get the CK's VC to work... what I would do is use Snip to copy over any new data from one file to another. It has yet to be proven that Snip4.2 corrupts data if you disable data compression (disabled by default) and use it with "hex mode" and "use new subrecord editor" checked (both in the Options pulldown).

Now if you want to know how to 'merge' your plugins with TESvSnip v4.2...
- backup all files, preferably using a 'version control' system (such as file numbering/appending or placing in different folders)
- open both your 'master' file (not necessarily an ESM) and the 'new' one to merge into it - both in one instance of Snip
- in the 'new' file, select the records you want to copy and press cntl-C (using the r-click menu has been buggy, I never have problems with cntl-C)
- go to the appropriate place in the 'master' file.. and cntl-V (again, r-click menu is buggy so don't use it EXCEPT when deleting records.. as I'm about to tell you)
- now select then drag each record to where the 'old' record is (placement may matter in some cases, I do it just in case); I suggest ALWAYS placing it above or below the 'old' - so you don't mix them up for the next step
- select the old records and r-click "delete" them (pressing the delete button has been buggy for me on occasion, r-click/del works every time)
- save the 'master' plugin (I actually save quite frequently, and completely shut the app down and restart it to prevent garbage collection drama)

= as for ESM/ESP splitting, I place everything in my plugin in one file (an ESP that's editable in the CK); then open it in Snip, delete everything I don't want in whichever file (eg-nilla stuff deleted from the ESM), then save the file using a different name (and tick the ESM flag in the header if need be). Then reopen the CK-editable file in Snip again, and delete everything you left in the file you just created; now each new file should have what the other file is missing (if copy-paste everything in one to the other, you'd get your CK-edit file again).

Here's a Dropbox link to my latest recordStructure.XML - it has TONS of decoded data (WAY more than any site or wiki I've found). I've been decoding every piece of data and updating it, and making it easier to read and more accurate descriptions. This file is also great for people working on their OWN 3rd party apps, and want to see the most complete set of data structures going... https://dl.dropbox.com/u/67168394/RecordStructure.xml. Even if you don't trust Snip to edit a plugin, it is AWESOME for looking at the data - and with this XML file, you can actually understand what most of it means!


ONAMs: In my recent data anolysis, I found that entries into the HEDR's ONAM subrecord are NOT required by all REFRs and ACHRs (seemingly these are the only record-types that use it).. even if those forms are overrides and change another ESM's data. I have yet to figure out EXACTLY what the criteria is to "make it on the list", but my best guess right now is if LCTN data is changed (which could be caused by a number of things.. such as editing a CELL, baseObject, a ref, the actual LCTN, etc). Another possibility is if enable/activate dependency has been changed. Otherwise, most EVERY override form in update.esm is not in the ONAM.. the ones that are on the list are all in three cells, and some of those forms aren't even CHANGED in the slightest (except timestamping).

You guys may be interested to know (if you don't already), that the CK will delete several certain types of subrecords - in a couple cases replacing them with other types. Either this is evidence of legacy data in skyrim.esm (leftover from Obliv/FO) being updated to a new format (probable), or that the public-release CK is destroying data willy-nilly in very specific/reproducible cases (possible given the number of bugs and hackage). Examples of this are in WATR records (3 NNAM subrecords for textures, when saved in the CK they are changed to NAM2/3/4 subrecords).

In LCTN records (which may be responsible for the entries into the update.esm's ONAM), all the subrecords beginning with "LC" are outright deleted (LCPR, LCUN, etc).. and not replaced. I think this data is now being stored elsewhere (like in an ACHR).. but I haven't gotten far enough to say for sure. If not, this could be the root of some people's problems in certain mods (but I doubt it).

[sorry for the long post, I haven't much time to check back and post lately.. so I get it in when I can heheh]

[EDIT: when copy-pasting/deleting stuff in Snip, only select stuff in the same grup/subgrup.. in other words, you can't select refs in two different cells and copy/paste them all at once. you have to copy one cell's refs, paste, copy the other cell's refs, paste, etc. You can select/copy/paste multiple grups, but only if they are all on the same 'level' - and keep in mind that EVERYTHING in the grup is copied. I suggest copying one grup at a time - as too much at once may cause memory-management issues and cause drama. Also, save frequently - and change those saved files' names each time to create a decent bank of backups should problems arise.

I've been using all these techniques with Snip for MONTHS on several mods, and have yet to have any problems - my mods are actually WAY cleaner than most, and don't have any issues that aren't caused by me (as opposed to Snip and/or CK data corruption).]
User avatar
BRIANNA
 
Posts: 3438
Joined: Thu Jan 11, 2007 7:51 pm

Post » Sat Nov 17, 2012 3:50 pm

Yeah you guys were right. It gives me this error when I try to check in:




Is there a way to get around this?

EDIT: Would it help to add MyMod.esm as a master file dependency via Wrye Bash?

Yes, that helps. Version Control refuses to add new dependencies to ESM's, do it manually in advance and it'll be fine.
User avatar
Benito Martinez
 
Posts: 3470
Joined: Thu Aug 30, 2007 6:33 am

Post » Sat Nov 17, 2012 11:39 am

I'm starting to feel stupid now. This is so ridiculous :(

-I tried the thing where I store changes to a new empty .esm which has my original mod as a dependency and the changes appeared in-game but, like last time: The dialogue didn't work.
-I tried the thing where I edit skyrim.esm with a hex editor but that didn't change anything.
-I also tried having update.esm and not having update.esm as a dependency. Nothing worked.

However when I save the changes as a regular ESP file and not merge the changes with an .esm file: The dialogue works fine.

I can't believe you guys are still trying to help me.. or were :P
User avatar
Beat freak
 
Posts: 3403
Joined: Thu Dec 14, 2006 6:04 am

Post » Sat Nov 17, 2012 8:59 am

huge quote

Thanks for the reply but I really don't understand most of what you're saying :s
User avatar
Quick draw II
 
Posts: 3301
Joined: Thu Nov 08, 2007 4:11 pm

Post » Sat Nov 17, 2012 4:18 pm

I can't believe you guys are still trying to help me..

We be stubborn [censored]es. :biggrin:

Sounds like the merging breaks something with the dialogue. I must admit I haven't merged dialogue yet, we haven't reached that stage yet with MERP. I also heard that one of the latest patches broke something about dialogue, but since I don't work with dialogue yet I haven't looked at all the details. You might want to look into that, it could be the issue here.

I think the merging procedure has been correct. Try fixing the broken dialogue after the merge, rather than retrying merging again and again.
User avatar
Margarita Diaz
 
Posts: 3511
Joined: Sun Aug 12, 2007 2:01 pm

Post » Sat Nov 17, 2012 5:23 am

Patch 1.6 broke editing of vanilla dialogue. Patch 1.7 fixes this, as long as you're using an ESP. New dialogue hadn't been a problem, at least in ESP files. Alexander has found though that the dialogue bug has broken everything he's done for dialogue in his mod, and he's using an ESM. Patch 1.7 has not fixed the problem for him. So there's something about dialogue in an ESM that behaves differently.
User avatar
Jonny
 
Posts: 3508
Joined: Wed Jul 18, 2007 9:04 am

Post » Sat Nov 17, 2012 11:11 am

Patch 1.6 broke editing of vanilla dialogue. Patch 1.7 fixes this, as long as you're using an ESP. New dialogue hadn't been a problem, at least in ESP files. Alexander has found though that the dialogue bug has broken everything he's done for dialogue in his mod, and he's using an ESM. Patch 1.7 has not fixed the problem for him. So there's something about dialogue in an ESM that behaves differently.

That's most likely the case here as well then.

Has Alex tried moving dialogue from his ESM to a "lower" ESP? Or rather, if using VC, inserting a new ESM above his current ESM, moving all except dialogue to the new ESM and converting the now dialogue-only ESM to an ESP? That method should keep all references intact.
User avatar
Steph
 
Posts: 3469
Joined: Sun Nov 19, 2006 7:44 am

Post » Sat Nov 17, 2012 11:27 am

What I can do is convert the mod into a .esm via wrye bash and VC works fine like that. Just not the dialogue of course. You guys think it's safe to start using Version control even though dialogue isn't working? Is it fixable later on that is?


Oh and another thing: My mod crashes my game when I try and load the .esp version. I have to convert it to .esm if I want to load it up in-game. I should have said that a long time ago I think :S
User avatar
WYatt REed
 
Posts: 3409
Joined: Mon Jun 18, 2007 3:06 pm

Post » Sat Nov 17, 2012 8:28 pm

That's most likely the case here as well then.

Has Alex tried moving dialogue from his ESM to a "lower" ESP? Or rather, if using VC, inserting a new ESM above his current ESM, moving all except dialogue to the new ESM and converting the now dialogue-only ESM to an ESP? That method should keep all references intact.
No idea what all he's tried, I'm hearing most of it second hand from one of his testers.

What I can do is convert the mod into a .esm via wrye bash and VC works fine like that. Just not the dialogue of course. You guys think it's safe to start using Version control even though dialogue isn't working? Is it fixable later on that is?


Oh and another thing: My mod crashes my game when I try and load the .esp version. I have to convert it to .esm if I want to load it up in-game. I should have said that a long time ago I think :S
There's no way to know right now how the dialogue thing is going to play out. I'd say it depends on how Dawnguard is shipped. If that comes down as an ESP then Bethesda won't have much motivation to make ESM files work right outside of whatever magic handling makes Update.esm behave.

If your mod crashes as an ESP, chances are you ran into the same size limit Alexander did.
User avatar
Amber Hubbard
 
Posts: 3537
Joined: Tue Dec 05, 2006 6:59 pm

Post » Sat Nov 17, 2012 7:07 am

Is it alright to sue VC to create everything BUT dialogue though? I understand if there is no way of knowing. I just wanted to make sure I asked that question.
User avatar
Phoenix Draven
 
Posts: 3443
Joined: Thu Jun 29, 2006 3:50 am

Post » Sat Nov 17, 2012 2:24 pm

If you want to do it that way, sure, but the dialogue will need to be a separate file worked on by one person because there's no other way to merge the content. Yet. Gecko is marching along toward that real soon now.
User avatar
Kelly James
 
Posts: 3266
Joined: Wed Oct 04, 2006 7:33 pm

Post » Sat Nov 17, 2012 11:22 am

How would I go about moving dialogue from an .esm to a new .esp?

Should I merge only the dialogue forms with an empty .esm via VC and then convert the .esm to an .esp via Wrye bash?
Should I tell CK to ignore all forms except the dialogue forms and then save?
Or is there a better way?

I've actually tried the "ignore all forms except dialogue" and it didn't seem to work. I'm going to try the version control method now.
EDIT: I tried the version control thing as well. Didn't seem to work.
User avatar
louise fortin
 
Posts: 3327
Joined: Wed Apr 04, 2007 4:51 am

Previous

Return to V - Skyrim