I don't think it'll become a swiss-army-knife NIF-repair tool. NIFs which are "bad" should be fixed, by a person. If the tool can not "understand" what's going on in a weird NIF it may do more harm than help. Up I reported some NIFs in real bad shape, maybe a TNB-block of 3k instead of 176k doesn't cash Oblivion but the results are probably not pretty ...

So, after the tool be able to rw almost all NIFs I've thrown at it, what would be the suggestion to track down corrupted NIFs? For obvious reasons I can not do this file by file (starting Oblivion 50.000 times is out of the question I'd say). It's also almost impossible to visit each and every place where those NIFs could occur. I do not believe though, that even if all 20.182 files in "Oblivion - Meshes.bsa" woud be broken, there are the same amount of distinct reasons of corruption. I hope we maybe have like 10-15 reasons max. So I'd be very glad if I could get individual NIFs named which either go out of shape, or which cause CTDs. It's not necessary to come up with tons of them. I'll track the reason down thoroughly and fix it, and possibly we'll have a fire-and-forget tool in a little while for real. Thanks.

Tool is still here (just updated):
http://paradice-insight.us/stuff/oblivion/NIFopt.7z

I'll play around with the updated tool later. I would like to request though that the optimization steps listed by Baphometal be considered since they're not trying to repair broken stuff, just streamline working stuff. Unless those are already in place?

I'm very interested in what techniques, if any, could be used to locate and identify corrupted NIFs. Regardless of optimization tools, some mods just distribute bad NIFs, and even PyFFI with all the latest bells and whistles breaks things, as I'm only recently learning.

Edit: I wonder if there's a way to script a special cell to instantiate all meshes in sequence or something? I'll try looking through OBSE commands, but while I have background in software development, I so far have little experience with making Oblivion mods.

I have no problems implementing straightforward things, but I'm very hesitant to implement some of the highlevel "optimizations" before I/we know they are not possibly the cause for PyFFIs broken NIFs ...
I had to implement (I was forced to because of the incoming NIFs, otherwise I'd have to program monstrous amounts of special code):
- clearing empty controllers
- clearing empty properties
- clearing empty children/extradata
- correcting texture-paths (to allow BSAing)
- removing geometry-nodes without geometry (<2 vertices, <1 face)


Every time I detected some real bad NIF I tried to record the fault and output some clear-text about it. So whenever you find another NIF with that fault you'll be adviced.
On the other hand, a NIF which comes out of the tool broken, is because of a bug. I just want to find a way to efficiently track those down, otherwise it's going to be an endless story ...

BTW: Updated the tool with texture-path fix. OOO had enourmous numbers of "/texture/..."-paths (1/3 of the files approx.), and I wanted to BSA it.

Excuse my ignorance, but could I get some quick instructions on how to run this? The NifOpt program?

If you just call the program in a shell it'll tell you. It can be as easy as "nifopt myif.nif", then it creates a 'mynif_opt.nif'
It doesn't do anything big, just copy basically. It gets more exciting with more parameters.

I detected that OOO has several "orphan" EGM-files. In that sense that the NIF isn't present. I assume the NIF could be in the Oblivion - Meshes.bsa. Though this is extreme nasty, when a mod replaces only EGMs and the related NIF has been stripped of some vertices they go definitely broke. I haven't seen that before, but if someone starts doing the same this entire effort is pointless.

I almost have the impression OOO should receive a thorough verification. Broken NIFs, wrong texture-paths, "orphan" EGMs ... wonder what else is coming up.

I'm a little confused about the relationship of the NIFopt tool Ethatron is developing to PyFFI. Do they do the same thing? Should I run one after the other, and if so, in which order? Is PyFFI deprecated in favor of NIFopt (is that why NIFopt has taken over its thread)?

http://niftools.sourceforge.net/wiki/NifTools:

NIFopt is a front-end for niflib, PyFFI-spells are front-end for PyFFI. Just other programming-language, C++ vs. Python.
I'm not so sure it is relevant to people if it's "Py"thon or C++. The code-base in either case is the same, from amorilia and lot's of other people.

I'm sorry for the possibly "newb" question, but how can one use Pyffi to edit a lot of nifs, not in the optimization sense? I have almost 200 nifs which I need to scale x18, and remove any bhkCollisionObject branches - could I use pyffi to automate this tedious process? I'm really inexperienced at using command line stuff, and I don't know any python, and it would probably take me as long to learn how to do it all myself as it would to click through and edit all the nifs by hand, so I would really appreciate the help!

Thank you!

I have the reappearance of an PyFFI issue from way back - this time in Nehrim - as seen http://i361.photobucket.com/albums/oo54/psymon11b/CaveExteriormesh.jpg.

The mesh for cave exterior called CEntranceMtnSnowLg02.nif will just not show up in game.

I had this issue over a year ago with Oblivion - may still have the screenshots ... yes http://i361.photobucket.com/albums/oo54/psymon11b/MissingEntranceafterPyFFI_2.jpg and http://i361.photobucket.com/albums/oo54/psymon11b/MissingCaveentranceafterPyFFI_1.jpg. It was always cave exteriors and even if I took the un-PyFFI'd mesh and placed as a loose file this would occur. My conclusion was that it had something to do with the meshes around it and not that mesh itself. Anyway the next time I ran the original meshes through PyFFI that issue went away.

I've used the newer optimized meshes for Nehrim for about a year or so and have not seen that issue - until today. Which is strange.

I tried again to place a loose original mesh in the data folder and get the same result.

Has anyone else had this problem and perhaps found and answer? thanks.

==========
[edit 1]
Oh wait I remember now - it had to do with leading spaces in the file names and not being able to move those over in explorer.

But then why am I not seeing this in Oblivion - only Nehrim?

==========
[edit 2]
Ahh now my memory is working. I have Vista OS - so could not prevent the leading spaces being truncated. Someone (I think Arthmoor) created a temporary patch called PyFFI-Optimized Vanilla Meshes Patch.esp and I think it's changes eventually got ported into the UOPS which is why it is not in my Oblivion load orders. What it does is change the file path to not use the leading space in the file name.

Good thing I keep an archive of this stuff. I still had the esp and after changing the master to Nehrim.esp it works and the cave entrances are back.

Strange that all this time playing Nehrim I hadn't encountered an entrance with this issue - until I venture to the North (today).

The problem is solved for me, but I don't think that esp is available publicly anymore. does anyone have a link to it so I can post it on the Nehrim thread?

Just out of curiosity, how much of a performance hit would it be if I chose not to PyFFI the non-static meshes (creatures, bodies, armors, weapons, etc)? I'm thinking of doing this to avoid certain bugs that are introduced by PyFFIing meshes, specifically NPCs having grand mal seizures and certain NPCs not being able to attack (hill giants and Bengal Tigers(?) from MMM come to mind). The most bugs I have seen relate to actors and everything related to them. Would Oblivion being less buggy warrant the lack of a performance gain?

Since bugs themselves can drop your performance, it's definitely a good idea to avoid bugs. Rather than trying to figure out which things you should and shouldn't PyFFI though, use the http://www.gamesas.com/index.php?/topic/1172029-relz-oblivion-pyffi-optimization-kit/ which handles all of that automatically. It has various conditional lists of which types of meshes to apply which types of optimizations to, and will probably give safer results than trying to figure it out manually.

New release is out:

https://sourceforge.net/projects/pyffi/files/pyffi/2.1.10/

It fixes a wide range of bugs, including a memory leak on Windows reported by Alphanos. It now also integrates the optimization kit, contributed by Ulrim and Arthmoor. Updating from 2.1.9 is strongly recommended!

Thanks to everyone who reported bugs and/or helped fixing them, in particular Arthmoor, Alphanos, Ulrim, Gratis_monsta, neomonkeus, Koniption, and K'Aviash.

I have used PyFFi for awhile now and just have some questions I have not found answers to. I am refering of course to the new release, of which I just downloaded.

1) From the changelog: "Integrated far nif optimization with the general purpose optimize spell". What do you mean by optimize spell, I hear it mentioned, but never know what it means. Does this "optimize" apply to "Optimize with PyFFi" right-click option? Or do I have to use the utility in the "toaster" folder to make use of the "optimize" spell?

2) Also, should I use the "Optimize with PyFFi" option, or should I use the utility in the Toaster folder to optimize meshes? Which is safer or provides better results?

3) Could not find a straight answer for this one in particular: Is PyFFi safe for use with Fallout 3? What folders in the Fallout meshes bsa should I avoid optimizing?

Won't let me edit my above post again. Just wanted to say thanks for the hard work on PYFFI as it has made my Oblivion experience more bearable and stutter free. Thank you for the "zombie" process fix especially, now I no longer have to optimize larger jobs in smaller chunks, and now should not have to worry about PyFFi crashing when I leave it running overnight.

Thanks, Iriodus. To answer your questions:

• For Oblivion, it is safest to use the batch file (i.e. toaster folder), as it skips nifs with known issues. However, if you want to help improving the optimization process, then use the right-click method, and report any problems you run into on the tracker.
  
• The shell_optimize.ini file decides what happens if you use the right-click method - at the moment it just optimizes everything (no skip list).
  
• For Fallout 3 I guess you can use the right-click method. However, be sure to report any issues on the tracker. I haven't had a Fallout 3 related bug report in a very long time, so either it is very safe, or nobody has bothered reporting the problems.

Thank you for the reply. I have used the right-click method for all my Vanilla meshes, SI and Knights as well, without any real issues, and I have been using PyFFi since 2.1.9 was released.

When I have Fallout 3 installed again, I will test the Vanilla and DLC meshes, and report any issues. I recall a thread or two on the FO3 forums mentioning CTDs upon entering worldspaces optimized with PyFFi, so if I find that I will post it here.

I'm having problems getting the Toaster (version 2.1.10) to work. I have Python 2.7.2 installed (through the Wyre Bash 295 installer). I unpacked the Oblivion - Meshes.bsa with OBMM to the "In" folder of the Toaster. I then try the oblivion_optimize.bat and a new window pops up and it hangs there at Part 1 "Running PyFFI optimize ..." . However, when I right click on a folder, for example meshes/rocks it seems to run just fine.

EDIT: Just noticed that the log file was increasing in size. So its actually working but isn't outputting the log file into the dos window like it does with the right click method.

EDIT2: Its finally done after about 3 hours (any way of speeding that up?) and the log files had the following errors.

Errors found Pass One:

Errors found on Pass Two

Errors found on Pass Three

Interesting. I'm not seeing any of these failures - I suspect I may have a slightly different version of Oblivion. Could you send me a few of the files so I can inspect this further?

Something I noticed about pass 3, which is the texture path correction pass. It's being performed on the "in" folder. If a mesh there is found to need this correction, won't that end up overwriting something that's been optimized from the first two passes? I think pass 3 should be conducted on the contents of the "out" folder instead.

You're absolutely right. Actually, it would not overwrite anything, because resume is enabled by default (which skips existing nifs in the out folder). It will simply do nothing.

Would this work for oblivion_optimize_03.ini (just off the top of my head)?

[main]spell = fix_texturepathfolder = ./out[options]source-dir = ./outoverwrite = True

[EDIT: did a minor fix to above ini file]

I'm not sure why there's a skip list needed for the fix_texturepath spell, so I'm omitting it from that stage.

Of course, what is really needed here are spell-specific skip lists, so we can do everything in a single stage, from in to out.

[EDIT2: actually, fix_texturepath is already part of the optimize phase, so I think the original ini file does the right thing: it fixes texture paths also on any nifs that failed beforehand; only improvement I can think of is that maybe the skip list should be removed from it?]

If it's in the 1st stage then is the 3rd stage even necessary at all?

Yes, if the skip list is unnecessary it may as well get removed too. A one-click process that handles everything in the correct order with the correct skiplist handling would be awesome since it's far easier to just use the right-click method on an extracted folder.

Ok. If the first stage has a skip list, and the 3rd does not, then it makes sense to keep it to fix texture paths on any previously skipped nifs.

https://github.com/amorilia/pyffi/commit/ea59b0d2be7e36f8b29853027f7c530fa7463c61

Amorilia - Which files do you need, the log files? If so, I can send them to you Monday. I also have the log files for SI, Knights, and all the DLCs. So can send those as well if you want them.