[RELZWIPZ]PyFFI: Python File Format Interface

Post » Tue May 08, 2012 8:23 pm

Thread updated on March 26th, 2011


http://www.gamesas.com/index.php?/topic/1158842-inforelz-pyffi-python-file-format-interface/



Post Update Headlines

PyFFI version 2.1.10 has gone stable.

Quick Links

http://www.gamesas.com/index.php?/topic/1158842-inforelz-pyffi-python-file-format-interface/
http://www.gamesas.com/index.php?/topic/1123756-inforelz-pyffi-python-file-format-interface/
http://www.gamesas.com/index.php?/topic/1112284-inforelz-pyffi-python-file-format-interface/
http://www.gamesas.com/index.php?/topic/1061938-info-relzpyffi-python-file-format-interface/
http://www.gamesas.com/?showtopic=1014947

PyFFI Thread 4 - too old and has been purged.
PyFFI Thread 3 - too old and has been purged.
PyFFI Thread 2 - too old and has been purged.
PyFFI Thread 1 - too old and has been purged.

http://pyffi.sourceforge.net/
http://www.python.org/download/releases/2.6.6/ (required to Run PyFFI)
https://github.com/amorilia/pyffi/downloads
http://sourceforge.net/tracker/?group_id=199269 (sub-trackers for bugs, requests etc.)
https://github.com/amorilia/pyffi (ie the earliest that you can see changes/additions that will be in the next release is here)
http://www.tesnexus.com/downloads/file.php?id=37463 A set of INI files and batch scripts designed for complete bulk mesh optimizations. Official support thread http://www.gamesas.com/index.php?/topic/1172029-relz-oblivion-pyffi-optimization-kit/
http://www.tesnexus.com/downloads/file.php?id=37548 For those who cannot or wish not to run the full optimization process. Official support thread http://www.gamesas.com/index.php?/topic/1173353-oblivion-pyffi-binary-patch-kit/ (Now integrated into PyFFI 2.1.10)
https://sourceforge.net/projects/pyffi/files/pyffi/2.1.10/



ESSENTIAL INFO FOR ALL USERS



What is PyFFI?

PyFFI stands for Python File Format Interface and was written almost entirely by Amorilia (Contributors: wz, taarna23, seith, tazpn, PacificMorrowind)

Latest Changelog

https://github.com/amorilia/pyffi/blob/python25/CHANGELOG.rst

Release 2.1.10 (10 October 2011)
==============================

Fixed bspline data methods to handle invalid kfs with missing basis data (reported by K'Aviash).
Fixed mass, center, inertia methods to deal with cases where shape is missing (reported by rlibiez, see niftools issue #3248754).
Fixed center calculation of bhkListShape collisions, and fixed zero division error when creating very small collision shapes (reported by Koniption, see issues #3334577 and #3308638).
Fixed shortcut to uninstaller in programs folder (reported by neomonkeus, see niftools issue #3397540).
Fixed geometry optimizer to handle cases where number of morph vertices does not match number of shape vertices (reported by rlibiez, see issue #3395484).
Merged ulrim's optimization kit, along with arthmoor's improved ini files.
Integrated far nif optimization with the general purpose optimize spell (requested by Gratis_monsta, see issue #3177316).
New shell_optimize.ini for configuring optimization as executed from Windows shell.
Allow .ini files that do not have a [main] or [options] section.
Fix Windows shell integration to point to the new shell_optimize.ini file (reported by rlibiez, see issue #3415490).
Fixed zombie process problem on Windows when a toaster was running with multiple jobs (reported by Alphanos, see issue #3390826).


FAQ

Q - Is it safe?
A - Yes; many Oblivion players and modders have run it on thousands of nifs with no problems. There is of course the potential for problems but if so as long as you made a backup it is relatively painless to revert some meshes. Of course there are some meshes that you should NOT run it on (any with egms is the main type) - if you do run it on them you will likely run into problems.

Q - Is it safe to PyFFI meshes that are already PyFFI'd?
A - While performance benefits will be negligible, yes, it is perfectly safe to run meshes that are already PyFFI'd through PyFFI again.

Q - Will running a mesh that had errors introduced into it by a previous version of PyFFI have those errors repaired by running a later version on the bad mesh?
A - No, PyFFI cannot repair any damage that has already been done, and you will need to run the vanilla version of the mesh through PyFFI again. Make sure you keep a backup of the default meshes BSA in case you run into any problem meshes.

Q - Why is PyFFI increasing the file size of my meshes?! Shouldn’t optimization be reducing their size?
A - As of PyFFI 2.1.6, much of the optimization work now occurs through triangulation (improves performance, but increases file size) rather than stripification (improves performance while decreasing file size).

Q - Does PyFFI support muli-core processors?
A - Yes, PyFFI will utilize multiple processor cores and hyperthreading if they are available.

Q – Does PyFFI support the 64bit version of Python?
A – No, PyFFI does not officially support 64bit Python, and using it with the 64bit version might cause problems.

Q - Is there a visual quality loss?
A - No, PyFFI removes duplicate and empty data from the nif file.

Q- What's the latest stable version of PyFFI?
A- As of October 10th, 2011, 2.1.10

Q- What's the latest in-development version of PyFFI?
A- Currently none, the latest Dev. version has gone stable.

Q- Are there any meshes that I should not put through PyFFI?
A- Yes, there are some meshes that can cause problems.

- Any meshes with FaceGen files attached to them (that is, any NIFs that are accompanied by an EGM or TRI mesh). These should not be optimized, or visual anomalies (E.G. tears in the model) might result.
- The Roothavok meshes. It is currently unknown why these don't optimize properly.

Q - I got a weird bunch of test in the command line/in the log file after running PyFFI, what does it mean?
A - Usually that would be in the form of something like this:
*** TEST FAILED ON C:\PyFFI_Op\XMSGroundFleshContainer01.nif ****** If you were running a script that came with PyFFI, then ****** please report this as a bug (include the file) on ****** [src="http://forums.bethsoft.com/topic/1172342-relzwipzpyffi-python-file-format-interface/http://sourceforge.net/tracker/?group_id=199269"]http://sourceforge.net/tracker/?group_id=199269[/url] ***

Q - Why am I not getting the option to run PyFFI when I right click the Oblivion_Optimize.INI?
A - You have a third party text editor (E.G. Notepad++) that is interfering with the drop-down options menu and preventing this option from displaying. This is fixed by uninstalling the interfering text editor.


It means that there is something about the nif that PyFFI doesn't know what to do with or something like that, or sometimes a bug in the spell. If you get that it means the file will not have been optimized and will have been left unchanged so no fixing is necessary to play with it. However if you can report that here or on the sourceforge tracker if you have a sourceforge account, it can probably be fixed. To be fixed we would need
A: to know what spell you were running
B: a copy of the nif or kf.

What is of interest for Oblivion modding right now is what it can do with nif/kf files:
It contains a script called niftoaster.py which is an python wrapper script that runs other scripts on files fed to it. Niftoaster can do a whole host of things to nifs/kfs including: optimizing models, fixing many different model errors, changing the collision type/material of nifs etc. etc. It is also highly extendable to do whatever you want.

One of the most important spells (the name for each of the scripts) is Optimize; it actually runs a whole bunch of the fixing and optimizing spells in tandem; for mod users it is really the only one you're interested in; and for a lot of mod makers it is also the only one your interested in.


Why Should You Care?

Really you don't have to; however, if you make meshes you should, do retexturing etc. or if you want to try and squeeze a little bit of extra performance out of Oblivion.

With the latest versions of PyFFI, performance increases can be quite substantial, so running optimization spells on your meshes (including the vanilla meshes packed in the BSA Archives) is highly recommended.

Installation

Notice for Windows Vista/7 Users: Do not install Python or PyFFI to the Program Files directory. UAC will make life miserable for you if you do.

Prerequisite:http://www.python.org/download/releases/2.6.6/
Currently only the 32bit version is supported.

(If using the Installer version)
-Download https://github.com/amorilia/pyffi/downloads (Windows Installer)
-Run the Installer and follow the onscreen instructions.

(If using the manual version)
-Download https://github.com/amorilia/pyffi/downloads (.ZIP Archive).
-Uninstall any previous versions of PyFFI
-Unzip the package to wherever you wish for it to go (Windows Vista/7 users, see notice above).
-Run the following command from the command prompt window:
cd C:\(Navigate to your PyFFI directory here)C:\python setup.py install
->Example:
cd C:\PyFFIpython setup.py install
(You need to have Python registered in your system’s environment variables for the python command to function properly)
-If you have setuptools installed, you can streamline the process by running this command line:
easy_install –U PyFFI

Uninstallation

(If using the Installer version)
-Run the PyFFI_Uninstall.exe

(If using the manual version)
-Delete your PyFFI Directory
-Delete the PyFFI folders found in the Lib\site-packages folders in your PyFFI 2.5 and Python 2.6 directories.

Using PyFFI

The scripts PyFFI uses are termed “Spells.” PyFFI provides several spells for you to use, the most important of which is “Optimize”. There are several other spells for mod developers to use, but the layman user will likely only be interested in the optimize spell.

There are far too many spells available to go over in one forum thread, so only the most important ones will be described here:

Optimize
Modify_makefarnif
Modify_texturepath
Modify_collisiontype
Modify_Scaleanimationtime
Modify_reverseanimation


INFO CONCERNING MOD USERS/PLAYERS AND DEVELOPERS



Spell: Optimize

This is where the brunt of PyFFI is located. The optimize spell removes duplicate/ empty/ uneeded data from your meshes, decreasing the workload for your GPU to render them. This can result in noticeable performance increases if you are to, say, PyFFI the entire vanilla mesh set.

How To:

http://tesivpositive.animolious.com/index.php?page=pyffi but for those who may just need a refresher, I'll give you the quick version:

First, you will need the meshes that you wish to optimize. These are usually found loose or packed into a Bethesda Softworks Archive (.BSA) file. If they are loose, it is best that you find the package for the mod whom the meshes belong to and extract them to a folder outside the Oblivion directory. It is not recommended to run PyFFI on the meshes folder already inside the data folder for Oblivion.

If the meshes are packed in a BSA, use a tool like Oblivion Mod Manager or BSA Commander to unpack the archive to a separate folder.

Once you have the meshes you wish to optimize, you have two options:

1. Right-click the meshes folder and select Optimize with PyFFI
2. Place the meshes into your PyFFI’s in folder (ex: C:\Program Files\PyFFI\utilities\toaster\in) and run PyFFI from the oblivion_optimize.ini

The former option optimizes the meshes and overwrites the originals. The latter option writes the new meshes to an \out folder so the original meshes are retained.

Further optimization for _far.NIFs (VWD meshes) can be called with the following batch script:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive opt_cleanfarnif --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

Once you are finished, gather up all the meshes into one spot (only really concerns users of option 2). If you the meshes were provided loose, rebuild the mod's package if you intend to use BAIN or OBMM installation so to include the optimized meshes. If you are optimizing a BSA, repack the BSA, and rename it to whatever it has been defined to be in your sArchiveList INI entry (if optimizing the Vanilla BSA) or to ensure your mod's .ESP file will allow the game to find it. Don't forget to reset the BSA timestamps, either, or you may find the vanilla meshes overriding your mod-added meshes!

Example of PyFFI in Action doing the Optimize spell:

reading C:\PyFFI_Op\Weapons\braided\claymorefine.nifchecking for duplicate source textureschecking for duplicate propertiesremoving duplicate and empty childrenoptimizing geometriesoptimizing block 'Claymore:0'removing duplicate vertices(num vertices was 732 and is now 615)recalculating strips(strip length was 1518 and is now 1188)(average strip length is 20.917508)stitching strips (using 103 stitches)recalculating tangent spacewriting C:\PyFFI_Op\Weapons\braided\claymorefine.nif...

Known Issues

Take heed that there are several meshes that are incompatible with the optimization process:

-Any hair mesh
-Roothavok meshes (the hanging roots you commonly see in caves)
-Any mesh with a .EGM association (commonly helmets)
-Any mesh with a .TRI association (teeth, eyes, etc.)

Using option 1, therefore, is not recommended unless you are certain none of the above meshes exist in the set you are optimizing. The oblivion_optimize.INI comes with a skip list that will tell PyFFI to skip the problem meshes during the optimization process, and it works pretty well, so if you are, for instance, optimizing the entire vanilla meshes BSA, it is recommended to use option 2.

Watch out for these meshes if you are doing a vanilla mesh optimization:
meshes\dungeons\caves\exterior\ centrancechighlandsm01.nif
meshes\dungeons\caves\exterior\ centrancegoldcoastsm01.nif
meshes\dungeons\caves\exterior\ centrancegreatforestsm02.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowlg01.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowlg02.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowlg03.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowsm01.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowsm02.nif
meshes\dungeons\caves\exterior\ centrancemtnsnowsm03.nif
meshes\dungeons\caves\exterior\ centrancerockmosssm01.nif
meshes\dungeons\caves\exterior\ centrancerockybeachsm01.nif
meshes\dungeons\caves\exterior\ centrancewestwsm01.nif
meshes\dungeons\caves\exterior\ centrancewestwsm02.nif

Their names contain leading spaces that it has been reported by Windows 7 64bit users get truncated if you move the meshes folder around after you’ve finished the optimization process. If you try to use those meshes in that state in the game, the game will report them as missing. Windows 7 64bit users should DEFINITELY watch out for this, and Windows 7 32bit and Windows Vista (all versions) users should be aware of the issue. Windows XP users are safe from this.

If you've used PyFFI before, previous to v.1.0.0 of PyFFI did not triangulate NifTriStrips with lengths less than 10. Unfortunately this causes Oblivion issues and while cause frame stuttering, and on low performance machines can virtually halt the game.

In addition, the 3dsMax NifTools exporter bloats the exported nif file with duplicate everything (or very nearly duplicate everything). This is a known problem, which I believe is due to 3dsMax and not directly to the NifTools.

Blender and Maya do not suffer this problem.

Though that does not mean that duplicate data will not appear in meshes made with Blender or Maya.

Cool technique (courtesy of Arthmoor and H2ODK):

If it helps anyone, this is my current Oblivion_optimize.ini file:
[main]; run optimize spellspell = optimize[options]; any patterns of files that should be skipped, being as conservative as possible; (without quotes, separate different regular expressions by a space); at the moment:; - skipping hair nifs (vertex ordering!); - skipping roothavok nifs (not sure why, investigating); - skipping any nif that is known to have an egm or tri associated with it;   find . -name "*.egm" -or -name "*.tri" | sed 'sX.*/XXg' | sed 'sX.tri$XXg' | sed 'sX.egm$XXg' | sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' | sort | uniq | xargs;   (not necessary if all egm files are included, such as in vanilla Oblivion,;   but some mods only include nifs without egm files, and this makes sure;   these cases are handled as well); note: you can safely remove this list if you are sure that you are; optimizing fully extracted folders onlyskip = (?i)(?
That nasty looking skip sequence at the end was something H2ODK came up with and works well, but it needs a slightly altered process to be sure everything gets done. That's where oblivion_optimize2.ini file comes in:
[main]; run optimize spellspell = optimize[options]; any patterns of files that should be skipped; regex expressions; only run on files with these patternsonly = (?i)gnd[.]nif$ (?i)meshes.clutter

When I do an optimization, I use two batch files. #1:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

#2:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize2.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

The first one takes ages to complete, the second usually goes by fairly quick because it's only looking for a specific set of info. What these do though is add a much more robust extra layer of "just in case" so that it REALLY won't touch any helmets and/or hairs and such.

I also have one other used specifically for cleaning any loose _far.nif files:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive opt_cleanfarnif --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

You may also notice that they all drop a log file when they're done. That's handy for quick-scanning for the word error or warning to see if any bugs cropped up. It also reduces the amount of spam from the handler messages.


PyFFI Patcher

http://www.tesnexus.com/downloads/file.php?id=37548

One of the many hurdles involved with the usage of PyFFI was the inability to redistribute the optimized meshes for the game for copyright reasons. This is now no longer the case with PyFFI Patchers. This separate utility allows for the user to instead download a patch file containing only the differences between the vanilla meshes and the PyFFI optimized meshes and simply apply the mesh patches to the old meshes.

The advantage? It takes a lot less time and is much less involved for a user to install these PyFFI Patches rather than run the PyFFI optimization process themselves.

The downside? These patch files will need to be redistributed every time PyFFI is updated to improve the optimization process for Oblivion's meshes. This will mean increased time spent on downloads.

Is it worth it? If you do not have the time available to invest in learning how to use PyFFI, or you wish to perform a one-off optimization of your meshes, then yes, it is worth it. If you wish to remain on top of the latest developments for PyFFI, however, it may be easier to simply run the optimization process yourself.

A fairly recent addition in the form of the PyFFI Binary Patch KIt has now been released, as well. This tool not only makes it easier to obtain the PyFFI optimized meshes by allowing the users to patch their own meshes, but it also makes it easier to create your own binary patch files, too. Here's how it is done:

With 'Oblivion - PyFFI Binary Patch Kit' you'd have to:
    place the original meshes inside 'in'
    place the optimized meshes inside 'out'
    open 'patcher.py' with a text editor and at the end change
    scan(s_dir, s_list)#scan(t_dir, t_list)scan(p_dir, p_list)#create_patch(s_list, t_list)apply_patch(s_list, p_list)
    to
    scan(s_dir, s_list)scan(t_dir, t_list)scan(p_dir, p_list)create_patch(s_list, t_list)#apply_patch(s_list, p_list)
    run the patcher and patch files are created inside 'patch' folder, recursively.

Many thanks to ulrim for his work on this. :)


INFO CONCERNING MOD DEVELOPERS ONLY



Spell: Modify_makefarnif

This spell will generate VWD meshes (_far.NIFs). It is called with the following batch script/command prompt:
 “C:\Python26\python.exe” “C:\Python26\Scripts\niftoaster.py” modify_makefarnif  --only _far.nif$ --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt” 


Spell: Modify_collisiontype

This lets you quickly (in seconds) and accurately and on as many nifs as you want at once, change to collision type to anything that is supported (and easy to add more if desired). A real boon for Max users until the nifscripts properly support non-static collision export. The format for calling it is batch file/command prompt of:
cd C:\Python26python.exe Scripts/niftoaster.py modify_collisiontype -a static "C:\meshes" (replacing as need the mesh path and the python path, and static with the type of collision desired)


Spell: Modify_scaleanimationtime

As usual real quick and easy method to: slow down or speed up animations. will change the speed of basically any animation type (not tested on bsplines yet) called by:
cd C:\Python26python.exe Scripts/niftoaster.py modify_scaleaniamtiontime -a 0.6 "C:\meshes" (replacing as need the mesh path and the python path and the number with the desired multiplier)


Spell: Modify_reverseanimation

As usual real quick and easy method to do something... in this case reverse an animation - ie make a close animation from an open animation for a door etc.
cd C:\Python26python.exe Scripts/niftoaster.py modify_reverseanimation "C:\meshes" (unteste for bspline type animations so far)


Spell: Modify_texturepath

lets you quickly (in seconds or minutes depending on number of nifs) and accurately and on as many nifs as you want at once, change the source texture paths from x/x/wall01.dds to /whateveryouwant/wall01.dds - a real boon when it comes to retexturing a whole tileset. The format for calling it is batch file/command prompt of:
cd C:\Python26python.exe Scripts/niftoaster.py modify_texturepath -a textures/PM/BloodyAyleid/ "C:\meshes"

(replacing as need the mesh path and the python path, and the texture path with the desired texture path).

Now I'm sure some of you are thinking hmm none of those do anything for me but I sure could use something that batch changed X... but I don't know any Python oh well. Never fear; if you want a script you can always ask - not promising it will get done, but significantly more likely than without asking (and if you don't believe me just ask DaMage).

As always if something isn't clear ask and someone knowledgeable will try to answer.
User avatar
Soku Nyorah
 
Posts: 3413
Joined: Tue Oct 17, 2006 1:25 pm

Post » Wed May 09, 2012 3:40 am

Hmmm... I use x64 Python and while problems aren't present.
User avatar
Stephani Silva
 
Posts: 3372
Joined: Wed Jan 17, 2007 10:11 pm

Post » Wed May 09, 2012 3:37 am

Has anyone made an optimized Oblivion - Meshes.bsa with PyFFI >= 2.1.7 ? I'm having trouble with the latest versions.

I don't know if there's much difference from that beta in the first post but you can find the final release of 2.1.7 here: http://sourceforge.net/projects/pyffi/files/pyffi/2.1.7/
User avatar
Cameron Garrod
 
Posts: 3427
Joined: Sat Jun 30, 2007 7:46 am

Post » Wed May 09, 2012 1:17 am

Just wanted to point out that the current beta at beta 5 :) It supposedly fixes the mod-added clutter falling through floor thing...

@ulrim:
Like I mentioned in the last thread, I'm currently running a fully PyFFIed Meshes.bsa (using 2.1.9b3 without the skiplist) and have yet to run across anomalies :) Although I don't recommend everyone to try without the skiplist... the beta versions are pretty safe for pyffication.
User avatar
Kaley X
 
Posts: 3372
Joined: Wed Jul 05, 2006 5:46 pm

Post » Wed May 09, 2012 12:57 am

I'm having problems with 2.1.9 Beta 4. Whenever I right-click on a meshes folder and choose "Optimize with PyFFI", the command prompt window opens and closes quickly, and the meshes folder does not get optimized. Any ideas as to why this is happening?
User avatar
Lauren Graves
 
Posts: 3343
Joined: Fri Aug 04, 2006 6:03 pm

Post » Tue May 08, 2012 3:07 pm

I believe I did a full optimize with 2.1.8, but it's been a long time since my last big optimization attempt. I am 100% certain I did a 2.1.7 optimization, though, and I'm having no problems with it.
User avatar
Angela
 
Posts: 3492
Joined: Mon Mar 05, 2007 8:33 am

Post » Wed May 09, 2012 3:56 am

Did anyone had any problems with http://www.tesnexus.com/downloads/file.php?id=36882?

I couldn't get it to work, it generated only empty folders.
User avatar
Jack Bryan
 
Posts: 3449
Joined: Wed May 16, 2007 2:31 am

Post » Tue May 08, 2012 1:05 pm

Just a quick note to let you the links for 2.1.7b5 under Installation in your opening post don't work.
User avatar
Marcus Jordan
 
Posts: 3474
Joined: Fri Jun 29, 2007 1:16 am

Post » Tue May 08, 2012 8:19 pm

I've figured my problem and have now a good Oblivion.bsa.

Tomorrow I'll update the pyffi kit with an improved interface and an important fix for the optimization of far.nif meshes.
User avatar
Mizz.Jayy
 
Posts: 3483
Joined: Sat Mar 03, 2007 5:56 pm

Post » Tue May 08, 2012 4:59 pm

I'm trying to figure out a way to tweak the color information on massive amounts of meshes, can pyffi do that?

I'm talking about NiMaterialProperty: the 4 color fields (ambient, diffuse, specular, emissive) and glossiness.
User avatar
sally coker
 
Posts: 3349
Joined: Wed Jul 26, 2006 7:51 pm

Post » Tue May 08, 2012 3:00 pm

I'm not quite sure where to report this... I'm having a problem with 2.1.9 beta 5 or the new PyFFI Optimization Kit (version 6). I've had a few cases of stuff falling through the floor, so I thought I would try the new version along with the optimization kit. Well the stuff didn't fall through the floor, but the new meshes didn't work either...

Here's the deal:

This is for my Imperial Furniture Renovated mod. The way this works is when the player buys a large piece of furniture, they are actually given a marker. The player drops the marker on the floor and clicks on it. The marker is replaced with a havoked version of the item. The player can drag the havoked version of the item to fine-tune the position. Then they click on it again and are presented with a menu. If they select "lock in position" from the menu, the havoked item is replaced with a static version. All that works fine. Now, normally the player can click on the static version and a menu will pop up with various options. That does not work with the newly optimized meshes. As soon as I remove the optimized meshes, the menu pops up as it's supposed to. Furthermore, with the optimized meshes, the collision is largely gone. I can almost walk completely through the object. However, it seems no matter where I click, the menu will not appear. Returning to the previous version of the meshes (which were optimized with 2.1.6) works fine. The collision completely surrounds the object as well.
User avatar
Patrick Gordon
 
Posts: 3366
Joined: Thu May 31, 2007 5:38 am

Post » Tue May 08, 2012 2:10 pm

I've been testing 2.1.8 and 2.1.9b5 and I find the same results. Collision box optimization must be removing it altogether.

This has nothing to do with the skip list on the kit. From my experience optimizing the meshes, PyFFI 2.1.6 worked best without these problems.

edit: actually, it has to do with it - seems I have to check for collision optimization because PyFFI is going bananas with that.
User avatar
Maeva
 
Posts: 3349
Joined: Mon Mar 26, 2007 11:27 pm

Post » Wed May 09, 2012 2:05 am

@andalaybay:
You should really start an artifact in https://sourceforge.net/tracker/?group_id=199269&atid=968813 so that amorilla can check your meshes (both original & pyffied) and possibly fix it :) My experience says she responds faster to posts there than here...

It'd be great if this could be resolved within PyFFI itself.
User avatar
Amy Gibson
 
Posts: 3540
Joined: Wed Oct 04, 2006 2:11 pm

Post » Wed May 09, 2012 3:46 am

@andalaybay:
You should really start an artifact in https://sourceforge.net/tracker/?group_id=199269&atid=968813 so that amorilla can check your meshes (both original & pyffied) and possibly fix it :) My experience says she responds faster to posts there than here...

It'd be great if this could be resolved within PyFFI itself.

Done. It's similar to Ulrim's problem as well. Also, it only happens with static meshes. The havoked meshes seem to be working fine...
User avatar
Raymond J. Ramirez
 
Posts: 3390
Joined: Sun Oct 14, 2007 8:28 am

Post » Tue May 08, 2012 1:54 pm

The last large scale optimization I did was with 2.1.7. I haven't really done much of anything yet on 2.1.8 and haven't even downloaded 2.1.9. Nothing has been out of place in my mad wanderings so far. I certainly haven't been able to walk through bridges or had stuff fall on the floor, but then I honestly don't recall what version I last optimized most of the mods I have installed with.
User avatar
Brian LeHury
 
Posts: 3416
Joined: Tue May 22, 2007 6:54 am

Post » Tue May 08, 2012 10:39 pm

I recently downloaded and installed 2.1.9b5, an upgrade from 2.1.8. Haven't yet created any meshes with it, other than one throw-away test run using both PyFFi and the new Optimizer. I had planned to redo my Oblivion meshes BSA (currently optimized with 2.1.7) as well as finally optimizing several mods that have thus far escaped treatment. But I'm seeing reports that 2.1.9b5 might be a bit "glitchy". Is that indeed the case? If so, what is the most recent fully stable and glitch-free version?

Thanks in advance!
User avatar
lacy lake
 
Posts: 3450
Joined: Sun Dec 31, 2006 12:13 am

Post » Tue May 08, 2012 4:31 pm

Just a quick heads up that the havok issue (collisions being removed in many if not most static objects) has been identified & fixed in 2.1.9-beta6 (just released). Thanks to Ulrim for reporting this on the tracker (indeed, please use it, I may not keep track of everything that is posted here!!):

https://github.com/amorilia/pyffi/downloads
User avatar
Big mike
 
Posts: 3423
Joined: Fri Sep 21, 2007 6:38 pm

Post » Wed May 09, 2012 4:25 am

PyFFI 2.1.9 beta 6 has fixed the collision problems I was having. At least the MOPP based static meshes are fine. I will need to continue testing with other kinds of static collision meshes.
User avatar
NeverStopThe
 
Posts: 3405
Joined: Tue Mar 27, 2007 11:25 pm

Post » Tue May 08, 2012 8:27 pm

OP update:

-2.1.9b6 changelog

-PyFFI Thread 4 has returned to the depths of Oblivion.

-Inclusion of a new headlines section so you can easily see the latest updates and so I don't have to waste space making posts like these. Why didn't I think of this before? :facepalm:
User avatar
Lewis Morel
 
Posts: 3431
Joined: Thu Aug 16, 2007 7:40 pm

Post » Tue May 08, 2012 10:54 pm

Not sure if this is an harmful issue or just an annoyance.

PyFFi version affected: v219b6 (and earlier today with 219b5)

ISSUE: "Mopper.exe has stopped working"

This is a new one for me. I ran in to it this evening first while optimizing my "TamRes - meshes.bsa" with the combo of PyFFi 219b6 and the new PyFFi Optimizer. The Optimizer batch file had been running for some time when two "Mopper.exe has stopped working" message boxes appeared in rapid succession. At that time CPU usage dropped from 100% to basically nothing. I clicked CLOSE to terminate the two stopped instances of Mopper. PyFFi immediately resumed processing. CPU usage returned to 100%. Mopper kicked in as needed for the remainder of the PyFFi run. So far as I know I received no error messages (though I'm not sure where I'd look for them.)

Thinking the problem might lie with the PyFFi Optimizer, I copied the untreated TamRes meshes into standard PyFFi and ran oblivion_optimize.ini. It behaved exactly like my earlier attempt with the Optimizer. Two "Mopper.exe has stopped working" messages appeared in what seems the same spots they had before, bringing progress to a standstill. Additionally, the PyFFi progress window stopped advancing. And as before, as soon as I terminated those two stopped mopper instances PyFFi kicked back in and completed its run, with mopper kicking in as needed.

This happened to me once early today with PyFFi v219b5 (and the Optimizer). I could not repeat the issue then so assumed it was a fluke.

Has any damage been done by mopper ceasing to function during a run? Have files been corrupted? I didn't see anything in the progress window to indicate corruption or an error of any sort. Is this a known ignorable issue with the latest versions of PyFFi, annoying but harmless? Is there a fix?

I again state that this occurs with PyFFi 219b6 (and before that 219b5), and those versions of PyFFi in conjunction with the Optimizer. Python is 2.6.6.

Any insight on this is most appreciated.

Thanks in advance!
-Decrepit-
User avatar
Channing
 
Posts: 3393
Joined: Thu Nov 30, 2006 4:05 pm

Post » Tue May 08, 2012 5:15 pm

Sounds like something I ran into with another mod once where an instance of a bad mesh caused mopper.exe to fail. Never found the cause because I couldn't find anything in the output log to track it down with.

The ones where mopper.exe failed won't have been written so you shouldn't need to worry about it. Although narrowing down which meshes are causing this may still be useful.
User avatar
nath
 
Posts: 3463
Joined: Mon Jan 22, 2007 5:34 am

Post » Wed May 09, 2012 2:34 am

@andalaybay: Thanks for confirming!

@decripit: As Arthmoor says, it is nothing to worry deeply about. Likely the mopper is failing internally for some reason regarding the geometry. The core functionallity of the mopper is closed source (unfortunately) so chances are fairly high that it cannot be fixed. But if you could post the nif or nifs that are affected by this on the bug tracker, maybe a workaround is possible. Link to bug tracker FYI:

http://sourceforge.net/tracker/?atid=968813&group_id=199269&func=browse
User avatar
Lizbeth Ruiz
 
Posts: 3358
Joined: Fri Aug 24, 2007 1:35 pm

Post » Tue May 08, 2012 2:29 pm

@andalaybay: Thanks for confirming!

@decripit: As Arthmoor says, it is nothing to worry deeply about. Likely the mopper is failing internally for some reason regarding the geometry. The core functionallity of the mopper is closed source (unfortunately) so chances are fairly high that it cannot be fixed. But if you could post the nif or nifs that are affected by this on the bug tracker, maybe a workaround is possible. Link to bug tracker FYI:

http://sourceforge.net/tracker/?atid=968813&group_id=199269&func=browse
That I will. Only, in this case I wouldn't know what to send. The BSA I optimized was quite large, and as Arthmoor mentions there's apparently no log entry to handily track down the mesh(es) in question. I'll try and remember the other mod that caused mopper to fail. It's mesh folder was much smaller. Might not be that difficult to pinpoint the culprit there.

If it means anything to you ... and it doesn't to me ... here's http://i307.photobucket.com/albums/nn303/Decrepit_Waste/ESIV%20errors/MopperExeStopped1.jpg at that point. Sorry for the somewhat blurry image. I needed to reduce image size to conserve online storage space.
User avatar
Laura Richards
 
Posts: 3468
Joined: Mon Aug 28, 2006 4:42 am

Post » Tue May 08, 2012 12:22 pm

I'd check the number of PyFFI jobs you were running. Then, take note of the last number of meshes optimized within that window. Perhaps it helps track down to where to start testing.

Then, isolate those meshes individualy and run PyFFI on them to check for the culprit. Much easier to narrow down.
User avatar
Amanda Furtado
 
Posts: 3454
Joined: Fri Dec 15, 2006 4:22 pm

Post » Tue May 08, 2012 11:09 pm

Hello,

I have some questions in regards to PyFFI and was wondering if someone could help me out. I am new to PyFFI and decided I wanted to try optimizing my meshes. I wanted to try the "Cool technique (courtesy of Arthmoor and H2ODK)", but I am bit confused on what I need to do with it. I understand I need to edit my Oblivion_optimize.ini file and replace the current spell with the new one, so I did. That seemed easy enough, but then I kept reading and I seen that I needed to create another .ini file called Oblivion_optimize2.ini and put the other spell that is listed under the "Cool technique (courtesy of Arthmoor and H2ODK)", so I did that as well.

The next part under the "Cool technique (courtesy of Arthmoor and H2ODK)" is where I am having trouble since I don't understand what I need to do. What do the lines below mean and what do I need to do with them?

When I do an optimization, I use two batch files. #1:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

#2:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize2.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

The first one takes ages to complete, the second usually goes by fairly quick because it's only looking for a specific set of info. What these do though is add a much more robust extra layer of "just in case" so that it REALLY won't touch any helmets and/or hairs and such.

I also have one other used specifically for cleaning any loose _far.nif files:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive opt_cleanfarnif --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"

You may also notice that they all drop a log file when they're done. That's handy for quick-scanning for the word error or warning to see if any bugs cropped up. It also reduces the amount of spam from the handler messages.



Another question I have is, do I need to keep uninstalling PyFFI and reinstalling it when the updates come out? From what it looks like, there are always a bunch of updates coming out.

Lastly, will I have to keep having to reoptimize my meshes everytime a new update or version comes for PyFFI?

Any help is much appreciated and thank you in advance!
User avatar
Siobhan Wallis-McRobert
 
Posts: 3449
Joined: Fri Dec 08, 2006 4:09 pm

Next

Return to IV - Oblivion