[RELz] SkyProc - Skyrim Processing Library Thread 3

Post » Mon Dec 16, 2013 6:32 pm

http://www.uesp.net/wiki/Tes5Mod:SkyProc


Java-based Skyrim Processing Library

What is SkyProc?

SkyProc is a Java library that offers easy to use Java API for importing, manipulating, and exporting mods created by Leviathan1753.

The SkyProc library offers Java programmers the power to create and edit objects that represent Skyrim mods and records. It is able to import mods, or even an entire load order, and give easy access to the records inside. Programmers can then make any changes they wish, and export a working Skyrim patch that is customized to every user's load order.

Its purpose is to facilitate third party creation of smart programs that create custom patches based on any given load order. For many mods, this will help reduce, or completely eliminate conflicts. It is currently quite functional and allows access to the most common GRUPs.


What can SkyProc be used for?

It can be used to make extremely smart and versatile patchers capable of adapting to any load order. It can be used for things such as:

  • Customization Patchers
    http://skyrim.nexusmods.com/downloads/file.php?id=7654is an example of a mod that uses SkyProc for customization capabilities. Instead of making several esps with varying hardcoded settings, the RL SkyProc patcher reads in an INI file, and adjusts the Realistic Lighting.esp patch to contain the custom settings defined by the user. This gives RL an unprecedented amount of customization for users.

    .
  • Addition Patchers
    http://www.gamesas.com/topic/1345186-relzwipz-automatic-spells/is an example of a mod that uses SkyProc to seamlessly add things to the world, without causing incompatibility issues. It reads in the NPC_ and SPEL groups from all the active plugins in a user's entire load order, it then loops over each, adding spells to NPCs based on the spells difficulty, and the NPC's spell skill stats.

    A comparable patch made by hand would take an extremely long time to make. In addition, that hard-coded patch would only ever affect Skyrim.esm. If a user wanted the same thing done to say, Midas, the modder would have to do a customized patch for Midas by hand as well. If Midas ever came out with a new set of spells, the modder again would have to adjust by hand.

    Doing this type of mod using a SkyProc patcher will work on any mod fed to it, no matter if it's Skyrim.esm, Midas 1.0, Midas 2.0. It will still read in the GRUPs, do its calculations, and hand out the spells accordingly. What this results in is a single patcher that works for all mods. If someone has only Skyrim.esm, it will work. If someone has Midas as well, it will not only will work with Midas, but it will add Midas' spells to Skyrim.esm NPCs, as well as NPCs from other mods.


    .
  • Preparation Patchers
    http://tes.nexusmods.com/downloads/file.php?id=37904
    is an example of a mod that uses SkyProc (in this case, an ancestral embedded version) to prepare the user's mod setup for in-game scripts. DLL is a spawn system overhaul that needs all the Leveled Lists to be organized in a special way, and have DLL-specific dummy entries with scripts attached to them. DLL comes in two parts. The first is a SkyProc-style patcher that processed the user's load order and creates a patch with all the Leveled Lists in proper format. The second part is a set of in-game scripts that take advantage of the preparation and offer much more dynamic spawning probabilities.

  • Mod Creation Patchers
    http://www.nexusmods.com/skyrim/mods/2933/
    adds many new weapons. Instead of creating each of the slightly different entries by hand a SkyProc patch was used to create the weapon records and set the values as needed. A mod that needs lots of changes that can be done in a determined way can be automated to save the author's time. The final mod does not need to include the patcher but its use can make tedious work much easier with less chance of errors creeping in.

There are probably countless other types of applications that can be made with SkyProc, as it loads in mods, and then lays them on a platter for you to do whatever you want with them.

The possibilities are really up to you as the programmer to flex your coding muscles and take advantage of the power SkyProc gives you.




For more information, documentation, and help tutorials, please visit:

http://www.uesp.net/wiki/Tes5Mod:Skyprochttp://www.uesp.net/wiki/Tes5Mod:SkyProc http://www.uesp.net/wiki/Tes5Mod:SkyProc

Previous threads: 1, http://www.gamesas.com/topic/1384676-rel-skyproc-skyrim-processing-library-thread-2/

What is happening with SkyProc?

Leviathan1753 has been busy with real life of late. Until he has time to catch up with modding I've stepped in to do what I can (hence me posting the new thread instead of him). The SkyProc library is quite well laid out so I've been able to make minor changes and bugfixes. I doubt I'll be able to add large additions like new record types but if you have a feature request or bug report let me know and I'll see what I can do. Many of the existing GRUPs have the data processed but not exposed to the API so if you need something I can probably do it quickly.

The latest official http://code.google.com/p/skyproc.

My changes have http://code.google.com/r/dhtynan-dev/. With the compiled jar on https://db.tt/6rHIcGAz.

My changes so far:

  • Fixed crash with BodyTemplate.Get/Set when passed FirstPersonFlags.NONE
  • Fixed issues with BODT/BOD2 subrecords acting strangely and writing garbage data. Type is now transparent to the patcher and it should just work.
  • Allowed parsing of YNAM, ZNAM, and NAM7 subrecords on Weapons. No access to the data yet but it won't crash.
  • Added Head Part get/set flag support. Added genenums.HeadPartFlags enum for use with them.
  • Fixed COED entries in NPCs getting scrambled on patching. Containers too since they share the subtype.
  • CONT.getScriptPackage()
  • HDPT.get/setHDPT_Type()
  • Various out of order subrecords

Known issues:

  • NPC_ subrecords ATKD and ATKE only handle a single instance instead of repeating pairs. I think I'm close to a fix.
  • Record length validation fails for some people with cleaned DLC (other times too?). Any info on this would be a big help.
User avatar
The Time Car
 
Posts: 3435
Joined: Sat Oct 27, 2007 7:13 pm

Post » Mon Dec 16, 2013 5:23 am

So ive looked at the debug overview file and it shows a few errors, something about bad mods but it points to skyrim.esm and some others so ill post it here along with any code that i currently have.

My current code:

import static skyproc.WEAP.WeaponType.Bow;
import static skyproc.WEAP.WeaponType.Crossbow;
import static skyproc.WEAP.WeaponType.Dagger;
import static skyproc.WEAP.WeaponType.OneHAxe;
import static skyproc.WEAP.WeaponType.OneHBlunt;
import static skyproc.WEAP.WeaponType.OneHSword;
import static skyproc.WEAP.WeaponType.Projectile;
import static skyproc.WEAP.WeaponType.Staff;
import static skyproc.WEAP.WeaponType.TwoHBluntAxe;
import static skyproc.WEAP.WeaponType.TwoHSword;

GRUP_TYPE[] importRequests = new GRUP_TYPE[]{
GRUP_TYPE.WEAP,
GRUP_TYPE.ARMO,
GRUP_TYPE.KYWD
};

FormID ArrowsBoltsID = new FormID("0200434B", "Categories Redone.esm");
FormID BootsID = new FormID("02001D8F", "Categories Redone.esm");
FormID ClothingID = new FormID("02001D90", "Categories Redone.esm");
FormID CuirassID = new FormID("02001D8E", "Categories Redone.esm");
FormID GauntletsID = new FormID("02001D8D", "Categories Redone.esm");
FormID HelmetID = new FormID("02001D8C", "Categories Redone.esm");
FormID OneHandedID = new FormID("020012C4", "Categories Redone.esm");
FormID RangedID = new FormID("02001D8A", "Categories Redone.esm");
FormID ShieldID = new FormID("02001D8B", "Categories Redone.esm");
FormID StaffID = new FormID("0200434C", "Categories Redone.esm");
FormID TwoHandedID = new FormID("02001D89", "Categories Redone.esm");

for (WEAP allWeapons : merger.getWeapons()){
WEAP.WeaponType wepType = allWeapons.getWeaponType();

switch (wepType) {
case Bow:
KeywordSet bowKeys = allWeapons.getKeywordSet();
bowKeys.addKeywordRef(RangedID);
patch.addRecord(allWeapons);
break;
case Crossbow:
KeywordSet crossbowKeys = allWeapons.getKeywordSet();
crossbowKeys.addKeywordRef(RangedID);
patch.addRecord(allWeapons);
break;
case Dagger:
KeywordSet daggerKeys = allWeapons.getKeywordSet();
daggerKeys.addKeywordRef(OneHandedID);
patch.addRecord(allWeapons);
break;
case OneHAxe:
KeywordSet oneHaxeKeys = allWeapons.getKeywordSet();
oneHaxeKeys.addKeywordRef(OneHandedID);
patch.addRecord(allWeapons);
break;
case OneHBlunt:
KeywordSet oneHbluntKeys = allWeapons.getKeywordSet();
oneHbluntKeys.addKeywordRef(OneHandedID);
patch.addRecord(allWeapons);
break;
case OneHSword:
KeywordSet oneHswordKeys = allWeapons.getKeywordSet();
oneHswordKeys.addKeywordRef(OneHandedID);
patch.addRecord(allWeapons);
break;
case Projectile:
KeywordSet projectileKeys = allWeapons.getKeywordSet();
projectileKeys.addKeywordRef(ArrowsBoltsID);
patch.addRecord(allWeapons);
break;
case Staff:
KeywordSet staffKeys = allWeapons.getKeywordSet();
staffKeys.addKeywordRef(StaffID);
patch.addRecord(allWeapons);
break;
case TwoHBluntAxe:
KeywordSet twoHbluntAxeKeys = allWeapons.getKeywordSet();
twoHbluntAxeKeys.addKeywordRef(TwoHandedID);
patch.addRecord(allWeapons);
break;
case TwoHSword:
KeywordSet twoHswordKeys = allWeapons.getKeywordSet();
twoHswordKeys.addKeywordRef(TwoHandedID);
patch.addRecord(allWeapons);
break;
}
}

My debug overview file:

[1] [OPEN DEBUG FILE] Opening Debug File
[2] [Run Location] Program running from: C:\Program Files (x86)\Steam\SteamApps\common\Skyrim\Data\.
[3] [EXCEPTION] java.io.FileNotFoundException: Files\BlockList.txt (The system cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at java.io.FileOutputStream.(Unknown Source)
at java.io.FileWriter.(Unknown Source)
at skyproc.gui.SUMGUI.loadBlockedMods(SUMGUI.java:475)
at skyproc.gui.SUMGUI.open(SUMGUI.java:370)
at skyprocstarter.SkyProcStarter.main(SkyProcStarter.java:64)

[4] [Status] My Patch version: 1.0
[5] [Status] Available Memory: 245MB
[6] [Status] Max Memory: 3632MB
[7] [SUM] Patch needed because no patch existed.
[8] [SUM] Patch needed: true
[9] [SUM] Starting patch because user pressed patch.
[10] [SUM] Window Closing.
[11] [Needs Importing] Needs importing because force patch was on or patch needed updating.
[12] [SUM] Patch needed: true
[13] [START IMPORT THREAD] Starting of process thread.
[14] [SPGlobal] Skyrim App data thought to be found at: C:\Users\JCarter\AppData\Local\Skyrim
[15] [BOSS] Looking for BOSS.
[16] [BOSS] Running BOSS.
[17] [BOSS] BOSS complete.
[18] [Importer] Starting import of targets:
[19] [Importer] WEAP ARMO KYWD
[20] [Importer] In mods:
[21] [Importer] Skyrim.esm
[22] [Importer] Update.esm
[23] [SPGlobal] Skyrim.ini believed to be in: C:\Users\JCarter\Documents\My Games\Skyrim\Skyrim.ini. File exists.
[24] [EXCEPTION] java.lang.NullPointerException
at skyproc.RecordFileChannel.openFile(RecordFileChannel.java:40)
at lev.LInChannel.(LInChannel.java:43)
at skyproc.RecordFileChannel.(RecordFileChannel.java:20)
at skyproc.SPImporter.importMod(SPImporter.java:514)
at skyproc.SPImporter.importMods(SPImporter.java:447)
at skyproc.SPImporter.importActiveMods(SPImporter.java:305)
at skyproc.gui.SUMGUI$ProcessingThread.run(SUMGUI.java:924)
at java.lang.Thread.run(Unknown Source)

[25] [ERROR] /=== ERROR!
[26] [ERROR] (==== File: SkyProcDebug/Asynchronous log.txt
[27] [ERROR] \=== Line: 3
[28] [ERROR] \== Message: Skipping a bad mod: Skyrim.esm
[29] [ERROR] \========================================>
[30] [ERROR] /=== ERROR!
[31] [ERROR] (==== File: SkyProcDebug/Asynchronous log.txt
[32] [ERROR] \=== Line: 4
[33] [ERROR] \== Message: skyproc.exceptions.BadMod: Ran into an exception, check SPGlobal.logs for more details.
[34] [ERROR] \========================================>
[35] [EXCEPTION] java.lang.NullPointerException
at skyproc.RecordFileChannel.openFile(RecordFileChannel.java:40)
at lev.LInChannel.(LInChannel.java:43)
at skyproc.RecordFileChannel.(RecordFileChannel.java:20)
at skyproc.SPImporter.importMod(SPImporter.java:514)
at skyproc.SPImporter.importMods(SPImporter.java:447)
at skyproc.SPImporter.importActiveMods(SPImporter.java:305)
at skyproc.gui.SUMGUI$ProcessingThread.run(SUMGUI.java:924)
at java.lang.Thread.run(Unknown Source)

[36] [ERROR] /=== ERROR!
[37] [ERROR] (==== File: SkyProcDebug/Asynchronous log.txt
[38] [ERROR] \=== Line: 5
[39] [ERROR] \== Message: Skipping a bad mod: Update.esm
[40] [ERROR] \========================================>
[41] [ERROR] /=== ERROR!
[42] [ERROR] (==== File: SkyProcDebug/Asynchronous log.txt
[43] [ERROR] \=== Line: 6
[44] [ERROR] \== Message: skyproc.exceptions.BadMod: Ran into an exception, check SPGlobal.logs for more details.
[45] [ERROR] \========================================>
[46] [Import Mods] Done Importing Mods.
[47] [SUM] Patch needed: true
[48] [Mod Export] Exporting My Patch.esp
[49] [ERROR] /=== ERROR!
[50] [ERROR] (==== File: SkyProcDebug/Export - My Patch.esp.txt
[51] [ERROR] \=== Line: 4
[52] [ERROR] \== Message: File header was correct, but there were no GRUPS. Validated.
[53] [ERROR] \========================================>
[54] [Consistency] Exporting Consistency file.
[55] [SUM] Closing program after successfully running patch.
[56] [SUM] Closing program normally from thread.
[57] [DEBUG] Closing Debug File. Time was: 9191

If you need any more info then let me know.

Thanks.

User avatar
CxvIII
 
Posts: 3329
Joined: Wed Sep 06, 2006 10:35 pm

Post » Mon Dec 16, 2013 4:51 pm

Hi there.

Not sure I got it all but a few things could be improved.
import static skyproc.WEAP.WeaponType.Bow;import static skyproc.WEAP.WeaponType.Crossbow;import static skyproc.WEAP.WeaponType.Dagger;...
Could be changed to:
import static skyproc.WEAP.WeaponType.*
But this is just cosmetic.
When you declare FormIDs like
FormID ArrowsBoltsID = new FormID("0200434B", "Categories Redone.esm");
you should just use the last six digits/letters:
FormID ArrowsBoltsID = new FormID("00434B", "Categories Redone.esm");
The first two digits (the load order) is determined by your patcher via the name of the .esp/.esm file

EDIT: Your patcher is started from a wrong directory as well. It should run from "Skyrim\data\SkyProc Patchers\YourPatcher\" and not directly from the data directory.
User avatar
Catherine N
 
Posts: 3407
Joined: Sat Jan 27, 2007 9:58 pm

Post » Mon Dec 16, 2013 9:49 am

Yeh netbeans did the imports when i wanted to use each weapon type so i left it but it makes sense to do it your way so ill change that.

I didnt realise that the first 2 digits were the load order so ill change that aswell.

ah right i thought i was suppose to run it from the data directory to get all the esm/esps.

Thanks for your help.

Edit: Ive Just checked after adding what you mentioned and it now works properly so thanks to you and DienesToo for all your help, this makes me very happy and now stress free lol.

Edit2: So ive managed to get ammo working properly based on the examples from the previous code but the last thing i need to do is sort armour by type but armour types are limited to Heavy, Light

and Clothing which i assume are their actual types. I assumed from this that i would need to find the armour pieces i need by keyword, those being armorBoots, armorCuirass and so on.

My question is, if im using a for loop to get all armour. How would i go about doing something like:

if armour has keyword (armourBoots)

add keyword (BootsID)

My attempts have failed so far. Thanks again for any help.

User avatar
lauraa
 
Posts: 3362
Joined: Tue Aug 22, 2006 2:20 pm


Return to V - Skyrim