[INFO] [RELZ]PyFFI - Python File Format Interface

Post » Sun Apr 25, 2010 10:32 pm

Thank you muchly :)

Gratis_monsta, based on http://www.gamesas.com/bgsforums/index.php?s=&showtopic=1061938&view=findpost&p=15517505 2.1.x is indeed the newest version of PyFFie. Forget that I said 2.0.6.


Thanks JdeRau for mentioning these meshes to not pyffie:
meshes/dungeons/misc/rootshavoc01.nif and related 02,03,04,05,06 an 07 nifs (I don't know if this has been fixed though)

I was not aware of these "special" nifs. Maybe Amorilia can confirm if they are still a problem or not. Otherwise, to skip them is probably safest, they are like only 8 skips out, will not hurt the game performance any big way.
User avatar
ashleigh bryden
 
Posts: 3446
Joined: Thu Jun 29, 2006 5:43 am

Post » Sun Apr 25, 2010 1:11 pm

PyFFI for multi core

use the option:
--jobs=X
X is a number >=2

exp. for 4 cores:
python.exe scripts\niftoaster.py optimize --noninteractive --skip=_far --jobs=4 v:\test2 > v:\test2\log.txt


exp. for far- nif meshes and 4 cores:
python.exe scripts\niftoaster.py modify_makefarnif --noninteractive --only=_far --jobs=4 v:\test2 > v:\test2\log.txt


Be careful, far-nif optimizing modifies texture- paths to texture\lowres- paths.
User avatar
darnell waddington
 
Posts: 3448
Joined: Wed Oct 17, 2007 10:43 pm

Post » Mon Apr 26, 2010 12:53 am

Hiya again folks :) I've spent the day pyffing my Oblivion Mesh.bsa and I must say the size reduction (an entire 150 MB for the first 5000 nifs) is startling!!! Looking forward to playing now...

Anyway, I noticed the automater failed to copy some nif's (3 in the FrostCrag DLC.bsa - magetower01.nif for example) and I could manually (by right-click) PyFFy them, but they give warnings like:
Pyffi.something.niftoaster: WARNING!!! NaN detected in geometry
Sorry for the kinda inaccurate warning ... but the main stuff is there :P

Would someone please tell me whether they're ignorable?? Or should I leave them like vanilla??
User avatar
Queen of Spades
 
Posts: 3383
Joined: Fri Dec 08, 2006 12:06 pm

Post » Sun Apr 25, 2010 7:39 pm

Hiya again folks :) I've spent the day pyffing my Oblivion Mesh.bsa and I must say the size reduction (an entire 150 MB for the first 5000 nifs) is startling!!! Looking forward to playing now...

Anyway, I noticed the automater failed to copy some nif's (3 in the FrostCrag DLC.bsa - magetower01.nif for example) and I could manually (by right-click) PyFFy them, but they give warnings like:
Pyffi.something.niftoaster: WARNING!!! NaN detected in geometry
Sorry for the kinda inaccurate warning ... but the main stuff is there :P

Would someone please tell me whether they're ignorable?? Or should I leave them like vanilla??

Hey Gratis, I stumbled over the exact same issue like you with the Alpha tool. After a lengthy discussion here in the forums (if you look long enough you can sure still find it as it was all like 3 weeks ago or so), it was suggested that basically that one should disregard those nifs that Alpha did not treat. I have left it at that. If Alpha would not treat or copy or whatever the one or other nif, it had its reason not to do so. Also in relation to the amount of nifs that got treatment, their number is negligeable (sp???). I would not be bothered too much, really :)
I am reluctant to use the right-click method as it does not take care of the nifs with EXM (or so) reference (that was under 2.0.5 I dunno if that's changed now with 2.1.x), so if Alpha tool would not treat it, I left it at that. And that seems to have worked quite fine I should say :D

Here is some of the discussion we had about it:
http://www.gamesas.com/bgsforums/index.php?s=&showtopic=1061938&view=findpost&p=15464447

There were some earlier posts of mine elsewhere but I'm too lazy to search for it now.
User avatar
christelle047
 
Posts: 3407
Joined: Mon Apr 09, 2007 12:50 pm

Post » Sun Apr 25, 2010 5:45 pm

I'd seen that post, and I must say they were most helpful :) I thought this NaN warning was something new to the 2.1.x versions so ... ah well :) Like you said, some 57 odd nifs outta a million probably won't make a difference!

PS: If you're talking about the .egm association thing with the rightclick method, it's automatically done from PyFFI 2.1.0 (according to the changelog) :)
User avatar
R.I.P
 
Posts: 3370
Joined: Sat Dec 01, 2007 8:11 pm

Post » Sun Apr 25, 2010 2:32 pm

PS: If you're talking about the .egm association thing with the rightclick method, it's automatically done from PyFFI 2.1.0 (according to the changelog) :)

Yes, .egm that's what I meant, and thanks for pointing this new feature out. That's handy to know :)
User avatar
Evaa
 
Posts: 3502
Joined: Mon Dec 18, 2006 9:11 am

Post » Sun Apr 25, 2010 1:36 pm

Upgrade/install/uninstall questions...

I've had PyFFi installed a while now, but other than make sure it worked never did anything with it. Mine is version 2.0.5. I of course wanna upgrade to 2.1.2 (already downloaded). But I use the version of Python that came with or was recommended by Wrye Bash - 2.5.2. Will this have to be upgraded too? If so, I suppose I need to uninstall all current Python stuff from within Windows Control Panel first, then install the latest version afresh? (The new doesn't simply update the old?) Will doing so mess up Bash?

Thanks!
-Decrepit
User avatar
Catherine N
 
Posts: 3407
Joined: Sat Jan 27, 2007 9:58 pm

Post » Mon Apr 26, 2010 2:59 am

Upgrade/install/uninstall questions...

I've had PyFFi installed a while now, but other than make sure it worked never did anything with it. Mine is version 2.0.5. I of course wanna upgrade to 2.1.2 (already downloaded). But I use the version of Python that came with or was recommended by Wrye Bash - 2.5.2. Will this have to be upgraded too? If so, I suppose I need to uninstall all current Python stuff from within Windows Control Panel first, then install the latest version afresh? (The new doesn't simply update the old?) Will doing so mess up Bash?

Thanks!
-Decrepit


Have you upgraded Bash to 2.77? It works very well with Python 2.6, and I think the latest features of Pyffi need Python 2.6.
But then I may be wrong. However, I'm running Bash 2.77, Pyffi 2.1.2, and Python 2.6. I have no problems at all using the tools.
User avatar
remi lasisi
 
Posts: 3307
Joined: Sun Jul 02, 2006 2:26 pm

Post » Mon Apr 26, 2010 12:57 am

Have you upgraded Bash to 2.77? It works very well with Python 2.6, and I think the latest features of Pyffi need Python 2.6.
But then I may be wrong. However, I'm running Bash 2.77, Pyffi 2.1.2, and Python 2.6. I have no problems at all using the tools.

I can confirm what Aellis says: While I am still using WB 2.76 and Pyffie 2.0.5, I do run them both on Python 2.6 and it works just fine!
User avatar
Jonny
 
Posts: 3508
Joined: Wed Jul 18, 2007 9:04 am

Post » Sun Apr 25, 2010 7:30 pm

Hi everyone. Sorry for the questions if they were already answered, little time to read all the threads... 4 and 5 have't any.
OBS: I'm using all stated in ( http://cs.gamesas.com/constwiki/index.php/Installing_Blender#Introduction ) so I think I'm up to date... All running in the x64 Win7, if it matter. For safety I use multiple backups for tests, so nothing wrong or lost til now. I'm using Pyffi automator, and my questions are:

1 - Some meshes end with bigger filesizes - is it normal, and really better to use these? or go back to the old smaller ones?
2 - It does try to chance everything, even the ones said as not recomended (creatures, etc). Can I use these in this new pyffi build?
3 - There are meshes, in nifscope, that have animations (real noob, sorry). Can they be pyffied or not recomended? (If no, then to open one by one is loooong, is there any other way to mark/remove then? or a list? example: \Meshes.bsa\meshes\oblivion\architecture\ruins\oblivionbridgeruingate
4 - Already heavy modded. Anything to worry? Pyffiing meshes from a mod already pyffied can cause problems? (Logic says no, but...)
5 - Repacking, is it better to leave umcompressed or to compress (and how much)?
6 - tnx in advance :)
User avatar
Amanda Furtado
 
Posts: 3454
Joined: Fri Dec 15, 2006 4:22 pm

Post » Sun Apr 25, 2010 2:18 pm

1 - PyFFI is designed to optimise meshes for performance rather than file size. This sometimes causes the NIF file to increase in size so yes, save these files for improved performance. You should also change the PyFFI automater to not test file size.
2 - Creatures and bows used to cause problems problems in older versions of PyFFI but they don't do so now. The only meshes where optimisation isn't recommended are:
Meshes with an EGM associated (mainly helmets and hair). From version 2.1.0, PyFFI is set to automatically ignore these files.
meshes/dungeons/misc/rootshavoc01.nif and related 02,03,04,05,06 an 07 nifs (reported in v2.0.5, not sure if it's fixed)
3 - Sorry, I can't answer this question.
4 - No problems. If you're using the same version as the modder, PyFFI will have nothing to do. If you have later version, newer optimisations will be added on.
5 - Uncompressed will reduce stutter caused by HD reads while compressed will save HD space. Go for uncompressed if you can spare the space.
6 - You're welcome.
User avatar
Bek Rideout
 
Posts: 3401
Joined: Fri Mar 02, 2007 7:00 pm

Post » Sun Apr 25, 2010 9:17 pm

Thanks IdeRau, helped a lot.
But how one of the question are still alive, and another has now beared a child, let's go, someone else might know...
3 - Asking again, (maybe already aswered by 2?) Can't bear to to make presunptions out of my mind... Then will put new examples for reference. But just a general rule, don't need a answer for each example, they are just to hel: If someone can open the oblivion meshes bsa, inside there are a subfolder called Oblivon. In there, multiple files with oblivion gate in name have animations. Istill, the oblivion harada plants, that like to atack you. And all the effect folder and the fire folder have animation too...
5 - Nice, can spare some. New HD bigger than I need. Now: can it be better to umpack/repack uncompressed every BSA then? Sorry, know it has not much with pyffi... but there are with optimization.
Til the next! :D

Edit: To be clear, I'm using Oblivion Pyffi Automator, not Pyffi Automation Utility alpha 5 - both are alright, right?
User avatar
chloe hampson
 
Posts: 3493
Joined: Sun Jun 25, 2006 12:15 pm

Post » Mon Apr 26, 2010 3:27 am

First, it's J, not I

I'll make a guess here and say that NIFs with an animation in it won't be affected by PyFFI. I say this because I haven't seen any complaints from anyone concerning animated meshes. Also, bows and creatures are animated but that problem was fixed so that might reinforce my assumptions.

I would say that uncompressing every BSA would yield even more performance gain. Note that the performance gain will only be seen when the contents of that BSA is loaded. For meshes and textures, this is when the cell is loaded (ie shorter loading screens and less stutter when travelling outdoors). For sounds and voices though, I think the gain will be when that sound is played. Note that when you recompress the BSA, you have to go into OBMM and reset the BSA time stamps or else the BSAs won't load properly.

Both automaters can be used to automate PyFFIs optimisation but the alpha 5 allows you to save the PyFFIed meshes elsewhere and keep the originals and it also provides support for multiple CPU cores (the Oblivion Pyffi Automator only supports one core). Otherwise, the Oblivion Pyffi Automator doesn't require any configuration to ensure it doesn't discard processed meshes.
User avatar
BrEezy Baby
 
Posts: 3478
Joined: Sun Mar 11, 2007 4:22 am

Post » Mon Apr 26, 2010 4:08 am

Thanks IdeRau, helped a lot.

I'll second that... thanks for the nice helpful answers to everyone... now for me to answer a question
But how one of the question are still alive, and another has now beared a child, let's go, someone else might know...
3 - Asking again, (maybe already aswered by 2?) Can't bear to to make presunptions out of my mind... Then will put new examples for reference. But just a general rule, don't need a answer for each example, they are just to hel: If someone can open the oblivion meshes bsa, inside there are a subfolder called Oblivon. In there, multiple files with oblivion gate in name have animations. Istill, the oblivion harada plants, that like to atack you. And all the effect folder and the fire folder have animation too...

animations should all work fine... morph animations used to fail but Amorilia coded around that some while ago and they work fine now.
5 - Nice, can spare some. New HD bigger than I need. Now: can it be better to umpack/repack uncompressed every BSA then? Sorry, know it has not much with pyffi... but there are with optimization.
Til the next! :D

soon PyFFI will be able to optimize the nifs inside of BSAs and save the BSA without you having to do unpacking and repacking... but for all bsas if your hd read speed is slower than your cpu's decompression you are better to have the bsa compressed and vice versa - most people are better to have it uncomppressed - but do make sure to defragment your disk after.
Pacifci Morrowind
User avatar
matt oneil
 
Posts: 3383
Joined: Tue Oct 09, 2007 12:54 am

Post » Mon Apr 26, 2010 2:58 am

Anyone willing to test this?

http://www.sendspace.com/file/jtljr6

Not much has changed in the optimizer (besides Pacific Morrowind's spell for deleting unused root blocks - a lot of mods seem to have these!), however the real new treat is some new features from the alpha tool that I've ported back into pyffi. Particularly worth mentioning are: multicore support, resume support, destination folder support.

Basically:
  • install it
  • go to Program Files\PyFFI\utilities\toaster in your file explorer
  • edit default.ini to reflect the path of where you store your oblivion nif files (or whichever files you want to optimize, for oblivion)
  • right click on oblivion_optimize.ini and select "Run with PyFFI"


You can edit default.ini to tweak various settings (you shouldn't touch oblivion_optimize.ini, it has exactly the settings that are supposed to work for everyone). The most interesting ones are:
  • jobs: number of cores to use
  • destdir: where the files end up (default is utilities\toaster\out)
  • resume: to change resume support (default is on)


Resume is implemented simply as "do not overwrite any existing files in the destination directory". You can interrupt the process at any time by pressing CTRL-C.

Feedback is welcome, particularly from those who are alpha tool users and still find their favorite feature(s) lacking.
User avatar
sarah simon-rogaume
 
Posts: 3383
Joined: Thu Mar 15, 2007 4:41 am

Post » Sun Apr 25, 2010 7:14 pm

Sounds very good indeed, Amorilia! I guess another round of pyffi-ing of vanilla meshes will be scheduled in the next days... :)
User avatar
cassy
 
Posts: 3368
Joined: Mon Mar 05, 2007 12:57 am

Post » Mon Apr 26, 2010 1:17 am

I'm going to give 2.1.3 a shot too.
It's about time to redo my meshes.
User avatar
I’m my own
 
Posts: 3344
Joined: Tue Oct 10, 2006 2:55 am

Post » Sun Apr 25, 2010 11:45 pm

I think a previous post I made got lost:
While I'm here, I have something to say to Amorilia and Pacific Morrowind. I finished running PyFFI on Fallout 3's main meshes a few hours ago. I split them between 2 folders and installed them with Wrye4Fallout. Starting the game with these meshes installed caused a CTD as soon as the game started. I then uninstalled the folder I thought was causing the problem. This time, the game reached the main menu but then CTD as soon as it has finished loading a game. I'm going to try find the meshes that are causing these problems sometime in the near future.


In the Sourceforge bug thread, I guess Amorilia was correct in saying that the optimise spell isn't designed for Fallout 3, to the point where FO3 meshes put through PyFFI's optimisations end up broken.
User avatar
Jacob Phillips
 
Posts: 3430
Joined: Tue Aug 14, 2007 9:46 am

Post » Sun Apr 25, 2010 6:16 pm

Indeed, Fallout 3 is sufficiently different from Oblivion to break things for sure. Eventually, I'll add Fallout 3 support for the optimize spell too - unless someone beats me to it of course. ;) I don't expect it to be very hard. It's just a matter of finding the time.
User avatar
Teghan Harris
 
Posts: 3370
Joined: Mon Mar 05, 2007 1:31 pm

Post » Sun Apr 25, 2010 8:50 pm

This has likely been answered before so for that I apologize, I did a bit of searching, but no luck.

With the newest build of PyFFI and the Automator, could I select it to PyFFI my "Meshes" folder at large, and then, if read correctly, it would handle all the meshes contained within all the numerous subfolders?

If true, how would the output handle this? would everything be arranged as it was before, and I could simply drag and drop back into data folder?

If it can do this out of the box, too good to be true!

Regardless I'll be giving this a try come next install.

Thanks to all of you for your hard work!
User avatar
Haley Merkley
 
Posts: 3356
Joined: Sat Jan 13, 2007 12:53 pm

Post » Sun Apr 25, 2010 6:26 pm

Meshes with an EGM associated (mainly helmets and hair). From version 2.1.0, PyFFI is set to automatically ignore these files.
meshes/dungeons/misc/rootshavoc01.nif and related 02,03,04,05,06 an 07 nifs (reported in v2.0.5, not sure if it's fixed)


Not to be rude to JdeRau, but can anyone confirm that PyFFI version 2.1.0 and above automatically ignore .egm files?
If so, this may be a good time for me to re-pyffi my files.

thanks for updates JdeRau, wouldn't of known about this had it not been for you post.
User avatar
Chloe :)
 
Posts: 3386
Joined: Tue Jun 13, 2006 10:00 am

Post » Sun Apr 25, 2010 11:01 pm

This has likely been answered before so for that I apologize, I did a bit of searching, but no luck.

With the newest build of PyFFI and the Automator, could I select it to PyFFI my "Meshes" folder at large, and then, if read correctly, it would handle all the meshes contained within all the numerous subfolders?

If true, how would the output handle this? would everything be arranged as it was before, and I could simply drag and drop back into data folder?

If it can do this out of the box, too good to be true!

Regardless I'll be giving this a try come next install.

Thanks to all of you for your hard work!

yes PyFFI maintains folder structure and everything as in - unless you tell it to save files to an alternate directory or something but just right clicking will just run through the folder and save in place.
its true!
Not to be rude to JdeRau, but can anyone confirm that PyFFI version 2.1.0 and above automatically ignore .egm files?
If so, this may be a good time for me to re-pyffi my files.

thanks for updates JdeRau, wouldn't of known about this had it not been for you post.

yes Amorilia coded that in for version 2.1.0 - see the changelog.
Pacific Morrowind
User avatar
Jessica White
 
Posts: 3419
Joined: Sun Aug 20, 2006 5:03 am

Post » Sun Apr 25, 2010 9:34 pm

yes Amorilia coded that in for version 2.1.0 - see the changelog.
Pacific Morrowind


Then it's not working right. I meant to report this sooner, but with 2.1.2 it did NOT ignore a mesh that has an egm file with it. It ended up getting optimized anyway. If it helps, all I did was right-click the folder and told PyFFI to optimize it. Then ended up seeing the helmet mesh get some verts chopped off. No idea if it's messed up though.
User avatar
alicia hillier
 
Posts: 3387
Joined: Tue Feb 06, 2007 2:57 am

Post » Mon Apr 26, 2010 2:05 am

What is the latest greatest...umm stable/reliable version I should be using for my meshes?

I am using Pyffi 2.0.5
and Blender 2.4.9b
User avatar
FoReVeR_Me_N
 
Posts: 3556
Joined: Wed Sep 05, 2007 8:25 pm

Post » Sun Apr 25, 2010 11:33 pm

@Amorilia
Take as long as you need. I personally think Oblivion needs the optimisations more than Fallout 3 since FO3 is more responsive to a CPU upgrade than Oblivion these days.

Still though, even some basic support for FO3 meshes would be nice to have, but like I said, take as long as you need.

@Metallicow
The latest version is 2.1.3, which can be http://www.sendspace.com/file/jtljr6. You can find more info about the 2.1.3 release http://www.gamesas.com/bgsforums/index.php?s=&showtopic=1061938&view=findpost&p=15619151. The latest stable one though is either 2.1.3 or 2.1.2. Take your pick.
User avatar
Ezekiel Macallister
 
Posts: 3493
Joined: Fri Jun 22, 2007 12:08 pm

PreviousNext

Return to IV - Oblivion