[.X chantelle .x Smith]

http://www.uesp.net/wiki/Tes5Mod:SkyProc

Java-based Skyrim Processing Library

What is SkyProc?


SkyProc is a Java library that offers easy to use Java API for importing, manipulating, and exporting mods created by Leviathan1753.
The SkyProc library offers Java programmers the power to create and edit objects that represent Skyrim mods and records. It is able to import mods, or even an entire load order, and give easy access to the records inside. Programmers can then make any changes they wish, and export a working Skyrim patch that is customized to every user's load order.


Its purpose is to facilitate third party creation of smart programs that create custom patches based on any given load order. For many mods, this will help reduce, or completely eliminate conflicts. It is currently quite functional and allows access to the most common GRUPs.

What can SkyProc be used for?

It can be used to make extremely smart and versatile patchers capable of adapting to any load order. It can be used for things such as:

• Customization Patchers
  http://skyrim.nexusmods.com/downloads/file.php?id=7654is an example of a mod that uses SkyProc for customization capabilities. Instead of making several esps with varying hardcoded settings, the RL SkyProc patcher reads in an INI file, and adjusts the Realistic Lighting.esp patch to contain the custom settings defined by the user. This gives RL an unprecedented amount of customization for users.
  
  .


• Addition Patchers
  http://www.gamesas.com/topic/1345186-relzwipz-automatic-spells/is an example of a mod that uses SkyProc to seamlessly add things to the world, without causing incompatibility issues. It reads in the NPC_ and SPEL groups from all the active plugins in a user's entire load order, it then loops over each, adding spells to NPCs based on the spells difficulty, and the NPC's spell skill stats.
  
  
  A comparable patch made by hand would take an extremely long time to make. In addition, that hard-coded patch would only ever affect Skyrim.esm. If a user wanted the same thing done to say, Midas, the modder would have to do a customized patch for Midas by hand as well. If Midas ever came out with a new set of spells, the modder again would have to adjust by hand.
  
  
  Doing this type of mod using a SkyProc patcher will work on any mod fed to it, no matter if it's Skyrim.esm, Midas 1.0, Midas 2.0. It will still read in the GRUPs, do its calculations, and hand out the spells accordingly. What this results in is a single patcher that works for all mods. If someone has only Skyrim.esm, it will work. If someone has Midas as well, it will not only will work with Midas, but it will add Midas' spells to Skyrim.esm NPCs, as well as NPCs from other mods.
  
  .


• Preparation Patchers
  http://tes.nexusmods.com/downloads/file.php?id=37904 is an example of a mod that uses SkyProc (in this case, an ancestral embedded version) to prepare the user's mod setup for in-game scripts. DLL is a spawn system overhaul that needs all the Leveled Lists to be organized in a special way, and have DLL-specific dummy entries with scripts attached to them. DLL comes in two parts. The first is a SkyProc-style patcher that processed the user's load order and creates a patch with all the Leveled Lists in proper format. The second part is a set of in-game scripts that take advantage of the preparation and offer much more dynamic spawning probabilities.

• Mod Creation Patchers
  http://www.nexusmods.com/skyrim/mods/2933/ adds many new weapons. Instead of creating each of the slightly different entries by hand a SkyProc patch was used to create the weapon records and set the values as needed. A mod that needs lots of changes that can be done in a determined way can be automated to save the author's time. The final mod does not need to include the patcher but its use can make tedious work much easier with less chance of errors creeping in.

There are probably countless other types of applications that can be made with SkyProc, as it loads in mods, and then lays them on a platter for you to do whatever you want with them.


The possibilities are really up to you as the programmer to flex your coding muscles and take advantage of the power SkyProc gives you.




For more information, documentation, and help tutorials, please visit:
http://www.uesp.net/wiki/Tes5Mod:Skyprochttp://www.uesp.net/wiki/Tes5Mod:SkyProc http://www.uesp.net/wiki/Tes5Mod:SkyProc



Previous threads: 1, http://www.gamesas.com/topic/1384676-rel-skyproc-skyrim-processing-library-thread-2/, http://www.gamesas.com/topic/1480921-relz-skyproc-skyrim-processing-library-thread-3/, http://www.gamesas.com/topic/1508900-relz-skyproc-skyrim-processing-library-thread-4/


What is happening with SkyProc?

Leviathan1753 has been busy with real life of late. Until he has time to catch up with modding I've stepped in to do what I can (hence me posting the new thread instead of him). The SkyProc library is quite well laid out so I've been able to make minor changes and bugfixes. I doubt I'll be able to add large additions like new record types but if you have a feature request or bug report let me know and I'll see what I can do. Many of the existing GRUPs have the data processed but not exposed to the API so if you need something I can probably do it quickly. The last version Leviathan released is http://code.google.com/p/skyproc but you should use the latest one I've released for your patchers. There are many important bugs that have been fixed as well as new features.



Big news! I moved SkyProc to https://bitbucket.org/DienesToo/skyproc-library. That will now be where both https://bitbucket.org/DienesToo/skyproc-library/src and downloads of https://bitbucket.org/DienesToo/skyproc-library/downloads will be. The SkyProc javadocs are now located at http://dienestoo.bitbucket.org/skyproc-library/. It also has an https://bitbucket.org/DienesToo/skyproc-library/issues?status=new&status=open and potentially a wiki. All bugs and feature requests will go there. While you are of course welcome to post bugs and requests here its best to assume if there isn't a ticket for something I've forgotten about it.


Change log:



Release 2.2.0.6:

• Work on making SUM up to date.


• Some optimization for memory use. Including updated Lev library included.


• Disabled writing detailed import logs for mods, set with SPGlobal.logMod(boolean).


• ActorValue.NONE/UNKNOWN changed to -1 as tesedit does.

Release 2.2.0.5:

• Added ability to create new TintLayers (for NPC_).


• Fixed bug with SubIntSigned.


• Added check for size of leveled lists and splits them as needed when exporting a patch.

Release 2.2.0.4:

• Fix for negative values in records being read and written incorrectly. This required a change to the Lev library so be sure to update your copy to the one included!


• Added SCRN to plugin header.

Older:

Spoiler

• Fixed crash with BodyTemplate.Get/Set when passed FirstPersonFlags.NONE


• Fixed issues with BODT/BOD2 subrecords acting strangely and writing garbage data. Type is now transparent to the patcher and it should just work.


• Allowed parsing of YNAM, ZNAM, and NAM7 subrecords on Weapons. No access to the data yet but it won't crash.


• Added Head Part get/set flag support. Added genenums.HeadPartFlags enum for use with them.


• Fixed COED entries in NPCs getting scrambled on patching. Containers too since they share the subtype.


• CONT.getScriptPackage()


• HDPT.get/setHDPT_Type()


• Various out of order subrecords


• Fixed NPC_ only processing the first ATKE/ATKD pair. Use get/add/remove/clearAttackPackages to access them now.


• ARMO added isTemplated()


• AMMO added AmmoFlag.NonBolt


• Condition getBaseActorValue moved from P_FormID to P_Int as its entry is the numeric index of genenums.ActorValue.


• False master plugins like the 2.0.0+ unofficial patches load with esms, as they appear in the users load order.

release 5:

• False esm comparator added.


• Records flagged Deleted will not export any subrecords.


• Condition - Use global properly exports FormID of global. CondFlag adjusted to work right.


• Various non-vanilla subrecords in AMMO, ARMO, WEAP.

release 6:

• Mods with false master esp's as masters won't throw missing master exceptions.

release 7:

• Various subrecords added and reordered.

Release 8:

• Added more subrecords. SkyProc.jar now includes its version in its manifest.

Release 9:

• Mod.getMajor will now properly check all GRUPs if none are specified.


• GRUPs now case insensitive for checking editor IDs. Effects anything backed by GRUPs like Mod.GetMajor(String edid).

Release 10:

• Reordered INFO subrecords to parse conditions correctly.

Release 11:

• Fixed conditions on INFO.


• Removed INFO.Response conditions and related functions since they shouldn't exist.

Release 12:

• Added missing MajorRecord.MajorFlags


• Added ARMO.get/setAlternateBlockMaterial() and updateBodyTemplate() which replaces a bodt to a bod2 and sets the armor's Non-Playable flag if needed.

Release 13:

• Changed BOSS behavior. SkyProc will no longer run BOSS before the patcher by default. If you want it to use the command line parameter -BOSS (-NOBOSS is now ignored).


• Fixed ARMO.get/setAlternateBlockMaterial() and ARMO.updateBodyTemplate() capitalization


• Made genEnums.Skill follow tes5edit format. If you used genEnums.Skill be sure to double check how you use it as it may have broken.


• Added WEAP get/setSkill(), get/setDetectionSoundLevel(), get/setAlternateBlockMaterial(), and get/setAlternateBlockMaterial()

Release 14:

• Added Mod.getQuests()


• Fixed WEAP.DNAM skills export


• Changed WEAP.WeaponType.Projectile to HandToHandMelee


• Added LeveledRecord.LVLFlag.SpecialLoot to enable LVLI to use the Special Loot formula


• Added LeveledEntry.get/setItemCondition() value of 1.0 is 100% health and 1.1 is 110%


• Fixed records without an EDID causing record length errors on export

Release 15:

• KeywordSet.add() won't add if the formID is already in the keywordSet.


• added command line argument -AllModsAsMasters to skip adding masters phases of exporting patch and use all imported mods.


• added SPGlobal.get/setAllModsAsMasters().


• Fixed order of false esm master files as masters of exported patches.


• Fixed getSkyrimAppData() adding an extra \\Skyrim if the user had to specify it.


• Added support for plugin specific ini files adding resource BSAs.


• Added GMST(String edid Int/Float/Bool/String) constructor to create GMSTs not in the skyproc enums.


• Added Mod.get/setAuthor().


• Fixed Owner.isValid() to allow itemCondition without an owner.


• Hopefully fixed QUST fragment scripts and alias scripts, needs further testing.

Known issues:

• Record fields not used in vanilla skyrim + dlcs are not always processed.


• Not all fields of supported records have access functions. The data is all there and almost all decoded so if you need something let me know and I can add as needed.


• SkyProc cannot automatically locate Plugins.txt on non-english Windows XP systems.


• When run through Mod Organizer old log files are not always cleaned up properly.

Development help:

Skyproc is a Java API so if you don't know java you should look up some tutorials, there are lots if you google for them. SkyProc of course requires a certain familiarity with Skyrim modding. SkyProc often presents information closer to how its actually stored in the plugin file format so comparing CK entries, tesedit entries, and SkyProc javadocs is often helpful to figure out how to accomplish something. The latest official javadocs are linked on the wiki, my updated versions are included in my compiled jars.



The wiki has some helpful tutorials with getting started and some basic actions. The skyproc javadocs include all the api info like class members and functions but is intended more as a reference. I've made the http://code.google.com/p/lootification/source/browse/ available since having examples is often very useful in figuring out details. You are also very welcome to ask questions here, in the past there has been a very helpful community providing assistance. For more in depth help feel free to message me and we can chat on gchat or steam or something.



Hopefully if you find something confusing you can update the wiki to be more helpful once you figure it out. I find I often forget what is confusing to a new comer once I've been digging in the guts of a project for a long time so my documentation is pretty sad.

[Jade MacSpade]

From previous thread:

That's your problem. You need to add changes to the patch skyproc is going to write. So patch.addRecord(). Also I noticed you don't have the line from the skyproc starter that actually adds the records from the loaded mods into your merger merger.addAsOverrides(SPGlobal.getDB()); if that's not just a copy and paste mishap then merger will be an empty Mod and you won't be able to get anything from imported mods.

I think you might have missed a confusing bit about skyproc and java. When you do Mod lootlists = new Mod("lootlists" , false); you are creating a new java object of type Mod representing a plugin with name lootlists.esp. But you could also do Mod otherLootLists = new Mod("lootlists", false); and have another java object of type Mod also with name lootlists.esp, but they are different objects. And both have no records in them since they aren't the Mod that skyproc imported for the file on your computer that is lootlists.esp. In the skyproc starter project it has

    Mod patch = SPGlobal.getGlobalPatch();

    Mod merger = new Mod(getName() + "Merger", false);
    merger.addAsOverrides(SPGlobal.getDB());

This does 3 thing. First it creates a java object called patch and assigns it the skyproc global patch. Skyproc automatically takes care of writing everything you add to the global patch into the plugin file once your patcher finishes running. Second it makes the Java Mod object merger with the name "skyproc starter merger", this isn't related to any plugin file and won't ever be written to disk, its only to help with bookkeeping. The third thing it does is add the global database of imported records to merger. That's all the records skyproc read from all your active mods (of the types in your import requests) as they would appear in game. Which is to say if you have a few plugins that modify the same record merger will contain the version as it appears in the final plugin to load, just like how it appears in game. In most cases you want to work with the records in merger since that's how they would appear in game. Obviously if you change them and add them to your patch those changes will take precedence in game since your patcher's plugin will load last.

Also you can either use code tags (code inside square brackets) to post short bits or post them somewhere like hastebin.com and then link it if its a longer piece.

[Amanda savory]

Ok, I'm back, and still confused with what I think is a simple process. FYI I've added in the code you suggested above, but there is still some confusion.

I have, among others, a leveled item with editor ID "SublistAmmoArrowDaedric" and form ID "0476AB" in a plugin with name "Loot Lists.esp". Now, how do I get its MajorRecord?

I've tried this:

Mod lootlists = new Mod("Loot Lists", false);
 
LVLI AmmoArrowDaedric = (LVLI)lootlists.getMajor("0476AB", GRUP_TYPE.LVLI);

You did say that this wouldn't work, and this does give a null LVLI. So I've tried using

LVLI AmmoArrowDaedric = (LVLI)merger.getMajor("0476AB", GRUP_TYPE.LVLI);

LVLI AmmoArrowDaedric = (LVLI)merger.getMajor("SublistAmmoArrowDaedric", GRUP_TYPE.LVLI);

And these both give null LVLI. So I'm still very confused on how to do this. This is literally the only thing I need to know to get this patch working, so help would, yet again, be appreciated.

And thanks again for the help you've given so far.

[Jeremy Kenney]

Ok, I've done some more testing, and I can add overrides in the patch to the ammunition and armor using the editor ID version of merger.getMajor. However, if I try getting any leveled list using getMajor it gives a null LVLI, even vanilla leveled lists. So, that's probably why I'm having issues.

[Jade]

Simplest question first is LVLI in your import requests?

And for using the FormID version you want to use the mod that the record originates from. So if its in skyrim.esm and also in lootlists.esp you need

FormID theListID = new FormID("0476AB", "skyrim.esm");
LVLI theList = (LVLI) merger.getMajor(theListID, GRUP_TYPE.LVLI);

If you just do merger.getMajor("0476AB", "skyrim.esm") its looking for a record with the edid "0476AB".

[Daniel Lozano]

It was not, and that fixed it.

*facepalm

It is always the simplest thing imaginable isn't it?

[benjamin corsini]

Okay, I'm back. I finally got through duplicating all the OTFT LVLI and replacing the entries with the duplicated armors. That seems to be working well. Now I'm going through and duplicating the non-unique npcs that wear playable armors. As a preliminary test, I've set up an array of strings contained in the appropriate RACE EDIDs so that the patcher can get the RACE of each NPC_ and compare it to the strings in the array. This way, I will only be duplicating the correct LVLN. Here is the code:

Spoiler

public class SMIALeveledNPCs {
        
    private final Mod merger;
    private final Mod patch;
    
    SMIALeveledNPCs(Mod m, Mod p){
        this.merger = m;
        this.patch = p;
    }
    
    String[] ArmRaces = {"Nord", "Breton", "DarkElf", "HighElf","Imperial","WoodElf","Orc","Redguard","Khajiit","Argonia","Dremora","Elder","Afflicted","Falmer","Temptress","Succubus","Draugr"};
    
    public void ConfigLVLNPCs(){
        for (NPC_ n: merger.getNPCs())
            if (!n.get(NPC_.NPCFlag.Unique)){               
                RACE r = (RACE) merger.getMajor(n.getRace(), GRUP_TYPE.RACE);                
                System.out.println(n.getEDID() + " " + n.getFormStr());
                    if (WearsPlayableArmor(r, ArmRaces)){
                        System.out.println(r.getEDID() + " 2");
               }                               
            }
        }
    
    
    private boolean WearsPlayableArmor(RACE r, String[] s){
        boolean b = false;
            for (String frag:s){
                if (r.getEDID().toUpperCase().contains(frag.toUpperCase())){
                    b = true;
                    break;
                }
            }
            return b;
    }

Anyway, I there is seemingly one NPC_ (the values of all the rest print to the console just fine) for which I get a NullPointerException when the compiler tries to getEDID of the race, and I can't seem to figure out which it is. I know it's the r.getEDID because I moved that command to a completely different place in the code and the Skyproc debug showed the error originating from the line to which I moved it. I've tried to place my System.out.println() s in such a way that the NullPointerException would print to the console at the point when when it actually happened so I could look for the Form in Tes5Edit to see if the RNAM (RACE) is null. However, either the nullpointerexception is not printed in the console at the correct time for me to use it to find the problem NPC_, or a missing RNAM value is not the issue. Has anyone seen this issue before? Is there something obvious I'm missing?

Also, I've noticed through all this that a lot of humanoid LVLN have RACE set to FoxRace. Anyone have any idea why this is?

[Adam Baumgartner]

A null pointer exception isn't a skyproc specific error but a general Java error. In this case it means you are trying to call a function getEDID() on r but there is nothing in r. You should always check that an object is non-null before you use it. When you do RACE r = (RACE) merger.getMajor(n.getRace(), GRUP_TYPE.RACE); r will end up null anytime merger doesn't contain a RACE with the FormID you gave it. This can happen if either n has an invalid formID for its race, say if someone made a mistake editing it in TesVEdit, or if the race is in a mod you didn't import, or if you didn't request that RACE be imported. The simplest way to deal with this is to check if r is null and print something if it is like

for (NPC_ n: merger.getNPCs())
    if (!n.get(NPC_.NPCFlag.Unique)){
        RACE r = (RACE) merger.getMajor(n.getRace(), GRUP_TYPE.RACE);
        if(r == null){
            System.out.println("Could not resolve race of " + n.getEDID());
        }
        else {
            System.out.println(n.getEDID() + " " + n.getFormStr());
            if (WearsPlayableArmor(r, ArmRaces)){
                System.out.println(r.getEDID() + " 2");
            }
        }
    }
}

Once you know what npc has the issue hopefully figuring out why you aren't getting the race won't be too hard.

You could also use the debugger to set a breakpoint for a null pointer exception which would let you inspect the variables when your code is running but that's a bit more complicated. But you should still always check that your objects are valid, especially since there are a lot of mods that were made before the CK that have garbage in them.

[dell]

I actually had something similar set where I was checking if r.getEDID was null, but I didn't think to do it for the race. I'll give it a shot. I mean, I've been trying to figure out which npc is the problem the way I described. I'll try it your way and I'll look into setting up a breakpoint for nullpointerexceptions. I don't know how to do that, but I'll figure it out. Thanks.

[Gemma Archer]

Okay, that worked. The error is from LvlChaurusAmbush [NPC_:0007932F] which is obviously from Skyrim.esm. Fortunately, this is not a leveled list of humanoids, so it doesn't really matter. Oddly, the RNAM value is populated in TES5Edit; it's another FoxRace. Is it possible having an RNAM set to FoxRace an indicator that the NPC_ record is not actually used (unless the NPC_ is actually a fox)? If that is not the case, going through and fixing those RNAMs for the humanoid races in my main esp is going to be somewhat time consuming. Oh, well...