Pyffi Script version alpha 5

What does this program do

Automates pyffi nif process for a folder (and any subfolders) that you select
optimized files are saved to a separate folder that you select
ini file has a skip_list - any file that matches a regexp from an entry in that list will be skipped.
any file with .egm associated with it is skipped
Information is displayed as the files are processed
If new file size is not less than [size_test] * original file size it is not saved. I added this because if a file doesn't loose at least a little size, it was probably already pyffied.
Allows the application to resume where it left off if you close the application (or your computer turns off/crashes/ect)


Download at:
http://www.tesnexus.com/downloads/file.php?id=25423

Alpha release please leave comments on forum page
http://www.gamesas.com/bgsforums/index.php?showtopic=1007148


If you have any suggestions for features, please leave comments on forum page



Programs needed
pyffi -> http://pyffi.sourceforge.net/
Python 2.5 - Python Installation package available at Wrye Bash Tes nexus -> http://www.tesnexus.com/downloads/file.php?id=22368


Development Resource
Autoit -> http://www.autoitscript.com/autoit3/


Notes!!!
When using the skip list please keep in mind that not all mods put nifs in a standard place. For example not every mod puts hair in the "characters\" directory. For this reason I suggest adding more detail to the skip list.
Example if you don't want to pyffi weapons.
Add "weapons\,"sword","axe","mace","bow" (and other weapon types) to your skip list.
You can also look through file_list.txt to see if there are any files that you might want to delete from the output.



Change log
alpha 5
* Improved the resume support. Now adds a file to the output directory that records all files that had pyffi successully run on them - whether or not they were copied over. You can Exit this application at any time and resume exaclty where you left off with no loss assumming you choose the same input and output directory.
* Removed the seperate armor, weapons, characters, creatures, fx, clothes selects. Everything is done through "skip_list".

alpha 4
*Quad Core support
*Add skip_list, any file that contains the characters found in skip list will not be processed.
Example :
skip_list = "fx\" , "custom\", "better cities", "hair\" - any file in fx, custom, or hair folder will be skipped. Any file with "better cities" in its name will be skipped
* Sudo resume function
If you stop the application (or computer crashes/shutdown) during the process. If you use the same source and detonation folders files that have already with optimized will be skipped. This has limited functionality as only files that were previously copied will be skipped. If they were not copied because they had an error or because they failed to meet minimum size reduction they will be processed again (probably with the same result).
* Many checks added to make sure that the program will run. If it can't it should give you a reasonable error message with what the problem probably is.

alpha 3
Added new entry to ini "size_test" if the optimized file is not less than this value times the original file it will not be copied over
Added new entry to ini "num_cores" use 1 for single core or 2 for dual cores. Quad core not supported yet (I need a quad core tester)
numerous bug fixes

Hi, I just tried I don't know what is wrong but it skipped everything. I know some meshes needs to optimized just targeted them but skipped really fast. I'm using Vista64, installed all required programs. Changed D: path to C: in .ini for phyton25 so need another tweak?

So, does this actually modify the meshes? Last time I tried pyffi, I left my machine running over night........

Thanks for quick reply
Yes there is tmp.log which is empty and temp.nif in Python folder.

Like I said its very fast only says no files copied - no size reduction and files skipped, other fields empty.

Here is ini file and logs.

ini:python_folder 	= "C:\Python25"skip_armor	= 0skip_weapons	= 0skip_characters	= 1skip_creatures	= 1skip_fx		= 1skip_clothes	= 1file_list:C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_1h.nifC:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_2h.nifC:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword02_1h.niffiles_skipped: EMPTY, thats interesting...output_log:No Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_1h.nifNo Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_2h.nifNo Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword02_1h.nif

Try right clicking on the program, and selecting "run as admin".

I suspect UAC is giving you some trouble. Another option may be to turn off UAC while trying to run the program.

Thanks for quick reply
Yes there is tmp.log which is empty and temp.nif in Python folder.

Like I said its very fast only says no files copied - no size reduction and files skipped, other fields empty.

Here is ini file and logs.

ini:python_folder 	= "C:\Python25"skip_armor	= 0skip_weapons	= 0skip_characters	= 1skip_creatures	= 1skip_fx		= 1skip_clothes	= 1file_list:C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_1h.nifC:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_2h.nifC:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword02_1h.niffiles_skipped: EMPTY, thats interesting...output_log:No Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_1h.nifNo Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword01_2h.nifNo Copy  - C:\Program Files (x86)\Bethesda Softworks\Oblivion\Data\Meshes\AdonnayCW\Bastard Swords\bsword02_1h.nif

I never use UAC and tried admin mode but still same. I notice this now on C: directory there 2 file; Python25temp and Python25tmp but not in phyton directory.

yea I need more error checking on the ini file.
Damn converting my private tools for public use is a bit messy. Sorry for the trouble.

First: only *_far.nif files

The thread: http://www.gamesas.com/bgsforums/index.php?showtopic=983713&hl=raevwd

The things to do (information from Arthmoor):
1 Collision data removed. If actual collision grids were still present, the branches have been deleted entirely. It's impossible to collide with an LOD object.
2 Animation controllers removed. It is impossible to animate an LOD object, so this data is bloat.
3 Geometry morphers removed. If a mesh had these, they've been removed, as they are only useful in animation.
4 All NiStringsixtraData blocks removed. These are nearly always associated with collision.
5 All BSX Flags removed. These are nearly always used for collision and animation.
6 All NiSpecularProperty blocks removed. Leaving these in place causes visual anomalies.
7 All NiBinaryExtraData blocks (tangent/normal spaces) removed. This data is not required for LOD meshes and helps tremendously in reducing memory usage.
8 All parallax flags turned off. The distances involved make this useless anyway.
9 All meshes have been assigned lowres textures instead of hires ones.
10 All NiAlphaProperty blocks removed, as leaving them leads to colored blobs.
11 Optimization using PyFFI after the above has been done.

How to do:
Use the power of spells - some.
we know: optimize
new:
-> fix_deltangentspace <- removes NiBinaryExtraData blocks
-> fix_disableparallax <- parallax flags turned off

I did the rest of the "things to do" list with NifSkope (I don't know how to do this with the nifoaster script), but perhaps it is possible to do this or a part (1,2,3,4,5,6,7,8,11 / 10 sometimes) of this in an automatic and much faster way.

>>2. I want to know which nif- file looks deformated after optimizing,... (this is much work, a blacklist will speed up these things - but it depends on the correct nif- file and the used pyffi version)
>I was thinking about adding a "modder mode" where before and after nif were displayed with nifscope and the user chooses to save or not save the changes. A bit more work so I'll only do this if there is interest.

A way to go, another:
1 before optimizing ask the user if he/she wants to use the blacklist
2 then generate the file list and remove the files which are in the blacklist and make a log to show which files are removed
3 optimize the files from the cleaned list

The blacklist is a solution, which needs information about nif- files from the community.

Perhaps two options for the ini file. First: use automatic file comparison = 1/0, Second: percentage were to choose new or old file = 0.00 ... 1.00
It is not only the size, which is optimized with pyffi (texturepaths, tangentspace, vertices, ...) - perhaps a little bigger file is better for the game.

does this program take full vantage of multicore CPUs? (nevermind I see that you're working on it )

A few ideas

Automation for BSA files open> extract> process> save .bsa (in a folder users choosing)

an indicator whether a file has been Pyffied( like if I tried to process All Natural.bsa witches all ready Pyffied{ I think })

the ability to save your work and resume processing at a later date even if the program has been shut down.

This would be hard to implement but would it be possible to harness the power of NVIDIA GPU 88xx and higher to assist with the processing? like CUDA or using their physics acceleration. it's been done before, not just for image processing or video rendering.

just a suggestion ( or mabye i'm totally blind) but I didn't see a link on here to the file (but of course easily found it on Nexus), so maybe add a link.

I'm on Vista (32bit) so I'll happily test this out. I expect no problems with UAC off (which is always for me) but I would guess that with UAC on it could easily not be happy.
edit: ran it and it copied the files (just did five for a test run) fine and they all dropped about 8% in size (which is exactly the same as with just running PYFFI manually on them) so working good.
Pacific Morrowind