[INFO] [RELZ]PyFFI - Python File Format Interface

Post » Sun Apr 25, 2010 8:18 pm

@amorilia
Thanks :thumbsup:
and
:bolt: testing
User avatar
Tiffany Carter
 
Posts: 3454
Joined: Wed Jul 19, 2006 4:05 am

Post » Sun Apr 25, 2010 7:19 pm

I have a question: Is this compatible with Vista and Windows 7? I am talking about PyFFi and the Automation utility.
I ask this because I have tried both it in the not so distant past. The automation utility didn't work, and I saw virtually no file size differences after optimizing a large meshes folder.
User avatar
Alessandra Botham
 
Posts: 3440
Joined: Mon Nov 13, 2006 6:27 pm

Post » Sun Apr 25, 2010 2:55 pm

I have a question: Is this compatible with Vista and Windows 7? I am talking about PyFFi and the Automation utility.
I ask this because I have tried both it in the not so distant past. The automation utility didn't work, and I saw virtually no file size differences after optimizing a large meshes folder.

yes (and for PyFFI some developement is done on a Vista system -- mine) both are compatible but you may have to change security settings and or locations to run them to avoid UAC with Vista and Windows 7.
Pacific Morrowind
User avatar
Ludivine Dupuy
 
Posts: 3418
Joined: Tue Mar 27, 2007 6:51 pm

Post » Mon Apr 26, 2010 3:24 am

Keeping both eyes on this thread :ninja:
User avatar
Jade Payton
 
Posts: 3417
Joined: Mon Sep 11, 2006 1:01 pm

Post » Sun Apr 25, 2010 6:46 pm

pyffi.toaster:INFO: ~~~ NiTriShapeData [] ~~~
pyffi.toaster:INFO: removing vertex colors


Am I reading right that this is telling the _far.nif spell to remove vertex colors from within the NiTriShapedata? That isn't necessary and should not be getting done because it will remove useful color data from a _far.nif. It's the NiVertexColorProperty blocks that need to be removed.
User avatar
Hayley O'Gara
 
Posts: 3465
Joined: Wed Nov 22, 2006 2:53 am

Post » Mon Apr 26, 2010 2:00 am

If I could make a suggestion, can the http://pyffi.sourceforge.net/contents.html please be updated sometime in the near future. At the moment, it is written for 2.0.5 and lists the unreleased 2.0.6 as still being in development (3 versions have been released since then).

Also, where can I find instructions on using some of the features Amorilia mentioned in a previous post:
...and a whole series of new command line options (akin the Alpha tool for nif optimization, not everything is there yet but more is planned!) for the toasters, including multithreading and a workaround for Python leaking memory when running the toaster for a long time (you'll need --jobs 2 or more for this to work), skipping files based on regular expressions, skipping nifs that have a .tri or .egm attached to them, and saving nifs in a destination folder different from the source folder.


EDIT: Another question. Does the PyFFI plugin for Blender and Maya automatically optimise the meshes when they are saved?

Help will be greatly appreciated.

Kind regards
User avatar
Farrah Barry
 
Posts: 3523
Joined: Mon Dec 04, 2006 4:00 pm

Post » Mon Apr 26, 2010 3:42 am

So what folders are safe to PyFFI now? With the 2.1.2 release?
User avatar
Laura Tempel
 
Posts: 3484
Joined: Wed Oct 04, 2006 4:53 pm

Post » Mon Apr 26, 2010 3:34 am

Am I reading right that this is telling the _far.nif spell to remove vertex colors from within the NiTriShapedata? That isn't necessary and should not be getting done because it will remove useful color data from a _far.nif. It's the NiVertexColorProperty blocks that need to be removed.

yes you are reading that right; I misunderstood there... I thought it wante both the NiVertexColorProperty and the vertex colour info from the NiGeomData removed.... I'll fix that up.
If I could make a suggestion, can the http://pyffi.sourceforge.net/contents.html please be updated sometime in the near future. At the moment, it is written for 2.0.5 and lists the unreleased 2.0.6 as still being in development (3 versions have been released since then).

oh yes... I'll mention that to Amorilia next pm I send. (though probably Amorilia will see this at the same time -- next time that he's online (I would guess this weekend)).
Also, where can I find instructions on using some of the features Amorilia mentioned in a previous post:

Oh nowhere right now... I'll try to get those all added to some of the docstrings for next time Amorilia updates the docs.
EDIT: Another question. Does the PyFFI plugin for Blender and Maya automatically optimise the meshes when they are saved?

Blender does (depending partially on export setting for some of it; such as whether identical NiTextureProperties are combined)... Maya I don't know since I don't use it (and don't even have it dled).
So what folders are safe to PyFFI now? With the 2.1.2 release?

should be pretty much everything... automatically skipping any meshes with associated egm files so will process almost nothing in the characters folder.
Pacific Morrowind
User avatar
Shae Munro
 
Posts: 3443
Joined: Fri Feb 23, 2007 11:32 am

Post » Sun Apr 25, 2010 8:54 pm

should be pretty much everything... automatically skipping any meshes with associated egm files so will process almost nothing in the characters folder.


Including when you're using the right-click on a folder method? If so, very nice. I've avoided touching the armors and characters folders for some time now because I was too lazy to move the .egm stuff out of the way. I'll probably end up waiting on these new vertex optimizations that have been mentioned before I do another round.
User avatar
kelly thomson
 
Posts: 3380
Joined: Thu Jun 22, 2006 12:18 pm

Post » Sun Apr 25, 2010 11:20 am

Including when you're using the right-click on a folder method? If so, very nice. I've avoided touching the armors and characters folders for some time now because I was too lazy to move the .egm stuff out of the way. I'll probably end up waiting on these new vertex optimizations that have been mentioned before I do another round.

correct... (same for if tri files exist for the mesh) rather great change that Amorilia did.
Pacific Morrowind
User avatar
candice keenan
 
Posts: 3510
Joined: Tue Dec 05, 2006 10:43 pm

Post » Mon Apr 26, 2010 3:30 am

Does PYFFI automatically skip nifs that include morph animations in them?

If not, I think it should:

I made an animated "root" bed for Blackmarsh. The bed starts out looking like a stump, but when activated "grows branches and leaves", that turn it into a usable bed. The problem I came across, is that when I PYFFI'ed the bed, the morph would not work right - many vertices were "welded" or something. I think I know the reason this happened, though: Instead of using a NiTransformController to scale a skinned version of the bed's root/leafy parts (which still might cause problems, not sure), I used a morph animation that makes those parts go from a scale of roughly "zero" to a scale of 1.0 (fully grown). So all the vertices are intially all bunched up together on a good portion of the bed. The initial 3D state of the bed in Nifskope is that of a stump.

I don't remember the version of PYFFI I had when this occurred, but I do know I have a newer version by now.

Yet, I think perhaps future ( if not any current vanilla ) morph animated nifs might end up with the same issue. If a person overlaps too many vertices close to each other, for whatever reason, the vertices could end up welded, then the morph might break.

Koniption
User avatar
jessica Villacis
 
Posts: 3385
Joined: Tue Jan 23, 2007 2:03 pm

Post » Sun Apr 25, 2010 11:03 pm

I have another suggestion for PyFFI mesh optimisation that I read about recently.

It involves changing bhkPackedNiTriShape/Strip collisions from non-MOPP collision types into MOPPed collisions. Unfortunately, I don't know much more about this since the thread I read didn't go into too much detail other than saying that this is a potential optimisation method.

So basically, is this a worthwhile addition to PyFFI? If you need more info, I can look for some more information in that regard.
User avatar
Sandeep Khatkar
 
Posts: 3364
Joined: Wed Jul 18, 2007 11:02 am

Post » Mon Apr 26, 2010 2:40 am

should be pretty much everything... automatically skipping any meshes with associated egm files so will process almost nothing in the characters folder.
Pacific Morrowind


Cool.
User avatar
:)Colleenn
 
Posts: 3461
Joined: Thu Aug 31, 2006 9:03 am

Post » Mon Apr 26, 2010 1:18 am

It involves changing bhkPackedNiTriShape/Strip collisions from non-MOPP collision types into MOPPed collisions.


I second this in a very big way. It's not a major issue with Bethesda's own stuff but it is a huge issue with mod-added meshes that end up with bhkPackedNiTriShape type collision. That collision type is horribly inefficient and leads to major frame rate issues. The current workaround for fixing it is to import into Blender and then export the mesh right back out, but obviously that's a high barrier for folks who can't deal with Blender.
User avatar
Kathryn Medows
 
Posts: 3547
Joined: Sun Nov 19, 2006 12:10 pm

Post » Sun Apr 25, 2010 2:15 pm

I'm having some problems with PyFFI. When I tried to PyFFI some Fallout 3 meshes, I got this error on some of the files:
http://img62.imageshack.us/i/error1pz.png/
http://img13.imageshack.us/i/error2jn.png/

It's looks like a write failure. I do have Windows 7 installed and my first thought was that Window's UAC was preventing the files from being overwritten. However, after temporarily disabling UAC I still had these errors (on exactly the same files as before). So clearly, the problem is the mesh rather than PyFFI's inability to save it's changes.
User avatar
Kelly Tomlinson
 
Posts: 3503
Joined: Sat Jul 08, 2006 11:57 pm

Post » Sun Apr 25, 2010 1:51 pm

I'm having some problems with PyFFI. When I tried to PyFFI some Fallout 3 meshes, I got this error on some of the files:
http://img62.imageshack.us/i/error1pz.png/
http://img13.imageshack.us/i/error2jn.png/

It's looks like a write failure. I do have Windows 7 installed and my first thought was that Window's UAC was preventing the files from being overwritten. However, after temporarily disabling UAC I still had these errors (on exactly the same files as before). So clearly, the problem is the mesh rather than PyFFI's inability to save it's changes.

hmm I see that... and yes it is a write failure; Is there a chance those two files are somehow locked to editing or something (that specific error message (AFAIK) should* only happen if the file can't be written)?
If not and a windows restart doesn't make it work and it is working on your other meshes (sounds like it is but not 100% sure from your post), please send those meshes for testing and figuring out what is up with them... also what spell were you running (Optimize?) (I can see that you were running opt_geometry but can't tell what else from those pics).
(responding to everything else shortly)
Pacific Morrowind
* of course 'should' isn't always the case
User avatar
Gisela Amaya
 
Posts: 3424
Joined: Tue Oct 23, 2007 4:29 pm

Post » Sun Apr 25, 2010 7:26 pm

Does PYFFI automatically skip nifs that include morph animations in them?

Yet, I think perhaps future ( if not any current vanilla ) morph animated nifs might end up with the same issue. If a person overlaps too many vertices close to each other, for whatever reason, the vertices could end up welded, then the morph might break.

Koniption

currently it does not skip for morph animations; works fine for all the bows and creature heads that i've seen... unless the vertices are exactly aligned (down to probably 4 decimal places I think) they shouldn't be removed... though it is certainly true that it could weld unfortunately... I'll leave what to do about this up to Amorilia... I'm just a humble secondary programmer after all.

I have another suggestion for PyFFI mesh optimisation that I read about recently.

It involves changing bhkPackedNiTriShape/Strip collisions from non-MOPP collision types into MOPPed collisions. Unfortunately, I don't know much more about this since the thread I read didn't go into too much detail other than saying that this is a potential optimisation method.

So basically, is this a worthwhile addition to PyFFI? If you need more info, I can look for some more information in that regard.

definitely seems worthwhile to me; will be much quicker to run a spell than importing and exporting with no changes for each individual mesh through blender.
Don't need more detail; I know the basics and I'm pretty sure it will be an easy spell to write up.
Pacific Morrowind
User avatar
Mr. Allen
 
Posts: 3327
Joined: Fri Oct 05, 2007 8:36 am

Post » Sun Apr 25, 2010 5:57 pm

hmm I see that... and yes it is a write failure; Is there a chance those two files are somehow locked to editing or something (that specific error message (AFAIK) should* only happen if the file can't be written)?
If not and a windows restart doesn't make it work and it is working on your other meshes (sounds like it is but not 100% sure from your post), please send those meshes for testing and figuring out what is up with them... also what spell were you running (Optimize?) (I can see that you were running opt_geometry but can't tell what else from those pics).
* of course 'should' isn't always the case

*Before I temporarily disabled UAC, I tried changing the folder permissions so that my user account was the owner of the files and folders and I changed it so that everyone had permission to have full control over those files (I can explain in greater detail if you don't know what I mean). That didn't work either. And before that, I changed all of Python's EXE files so that they run as administaters. I don't know if there is anything else I can do to give write permission.
*Restarting PyFFI and Windows doesn't work either.
*There were about a dozen meshes that gave me this problem. Every other mesh I optimised didn't have that error.
*Correct, I was optimising the meshes (by right-clicking on the folder).
*I have posted one of the problem meshes in the PyFFI bug tracker on Source Forge.

I also have a few more questions.
First, does PyFFI keep track of the meshes it has already optimised in a specific folder? This is one of the features of the PyFFI automater and is useful if you can't keep your PC running for the entire optimisation session.
Two, what happens if you try to optimise a mesh that PyFFI has already optimised (whether it is the same or an older version of PyFFI that optimised it)?
Last, what does PyFFI do with a mesh that has a flame or light node attached?
User avatar
Samantha Wood
 
Posts: 3286
Joined: Sun Oct 15, 2006 5:03 am

Post » Sun Apr 25, 2010 9:23 pm

*Before I temporarily disabled UAC, I tried changing the folder permissions so that my user account was the owner of the files and folders and I changed it so that everyone had permission to have full control over those files (I can explain in greater detail if you don't know what I mean). That didn't work either. And before that, I changed all of Python's EXE files so that they run as administaters. I don't know if there is anything else I can do to give write permission.
*Restarting PyFFI and Windows doesn't work either.
*There were about a dozen meshes that gave me this problem. Every other mesh I optimised didn't have that error.
*Correct, I was optimising the meshes (by right-clicking on the folder).
*I have posted one of the problem meshes in the PyFFI bug tracker on Source Forge.

well it certainly sounds like you did everything posible on that option - and since it was working for the others it does seem very unlikely that it was a write error... I'll (and likely Amorilia as well) have a look at that problem mesh.
I also have a few more questions.
First, does PyFFI keep track of the meshes it has already optimised in a specific folder? This is one of the features of the PyFFI automater and is useful if you can't keep your PC running for the entire optimisation session.

not yet - IIRC that and only saving if minimum size change x has occurred are the two features that aren't yet done.
Two, what happens if you try to optimise a mesh that PyFFI has already optimised (whether it is the same or an older version of PyFFI that optimised it)?

attempt to run; there is no specific sign that PyFFI has been run, except for that the file doesn't have any unnecessary data which -- if last run was with an older version there is even a good chance that it will have further optimization occur.
Last, what does PyFFI do with a mesh that has a flame or light node attached?

nothing special... just runs whatever spell you have specified.
Pacific Morrowind
User avatar
Etta Hargrave
 
Posts: 3452
Joined: Fri Sep 01, 2006 1:27 am

Post » Sun Apr 25, 2010 4:42 pm

Last, what does PyFFI do with a mesh that has a flame or light node attached?

nothing special... just runs whatever spell you have specified.
Pacific Morrowind

The reason I ask this is because Koniption mentioned to me that optimising a mesh with such a node attached might cause problems (didn't say what though). Is this correct?
User avatar
Nicola
 
Posts: 3365
Joined: Wed Jul 19, 2006 7:57 am

Post » Sun Apr 25, 2010 3:54 pm

The reason I ask this is because Koniption mentioned to me that optimising a mesh with such a node attached might cause problems (didn't say what though). Is this correct?


For the record, I don't know for sure that running PYFFI on flame and light nodes causes problems. I personally separate the flame and light nodes from the mesh, then optimize it, then insert them back in - just to potentially avoid problems. I do this, because a long time ago, I think I remember reading something (somewhere), about them causing issues. But honestly, I might be confusing the remembered info with something else.

I don't have that many flame and light node containing meshes that I need to optimize, so me doing this to the meshes, is no biggy.

But if anyone else reading this does know from personal experience that flame and light nodes cause issues with PYFFI, by all means do post in this thread.

Koniption
User avatar
Mari martnez Martinez
 
Posts: 3500
Joined: Sat Aug 11, 2007 9:39 am

Post » Sun Apr 25, 2010 10:36 pm

Hello to everyone :)

I'm extremely new to the modding scene and thought of trying out the awesome FCOM, but learnt about all the other neat stuff that could make Oblivion play nicely with all the new stuff - this PyFFI for instance! I've found the Alpha5 Automater and the BSA Commander, and have all my vanilla (GOTY) bsa's unpacked and sitting tight for pyffication :) I've also read up some basic stuff on what the PyFFI does and how to go about it.

It seems like some .nif's aren't meant to be PyFFIed, like animations and stuff. The automater provides a regex skipper that can skip all kinds of stuff, so I would love to know of any basic skip_lists to just PyFFI my vanilla bsa's (and maybe the mod bsa's like MMM, All Weather etc as well) If possible, I'll run it on my Data folder as well.

Tommy_H was gracious in telling me that only "hair" .nif's need to be skipped, while any stuff happening with the other .nif's can be safely ignored. I'm still skeptical about the animation thing (should creatures, weapons etc be PyFFied??) But I figure PyFFI got updated quite a lot. This thread mentions v2.0.6, but I found a v2.1.2 in sourceforge!

Any help would be great. Thank you :)
User avatar
Kayleigh Williams
 
Posts: 3397
Joined: Wed Aug 23, 2006 10:41 am

Post » Sun Apr 25, 2010 3:29 pm

Basically, the OP says that the only meshes that will probably cause problems are:
Meshes with an EGM associated with it (mainly open helmets and hair). I think the latest PyFFI automatically skips these though.
meshes/dungeons/misc/rootshavoc01.nif and related 02,03,04,05,06 an 07 nifs (I don't know if this has been fixed though)

Using the latest PyFFI on creatures and weapons is perfectly safe. Bows and creatures used to be a problem in older versions but that has now been fixed. As for PyFFI automation utilities, I personally prefer the http://www.tesnexus.com/downloads/file.php?id=20483.

While I'm here, I have something to say to Amorilia and Pacific Morrowind. I finished running PyFFI on Fallout 3's main meshes a few hours ago. I split them between 2 folders and installed them with Wrye4Fallout. Starting the game with these meshes installed caused a CTD as soon as the game started. I then uninstalled the folder I thought was causing the problem. This time, the game reached the main menu but then CTD as soon as it has finished loading a game. I'm going to try find the meshes that are causing these problems sometime in the near future.
User avatar
Marquis T
 
Posts: 3425
Joined: Fri Aug 31, 2007 4:39 pm

Post » Sun Apr 25, 2010 2:49 pm

Thank you muchly :)

PS: Why do you prefer the automater?? The only advantages I found (with respect to the newer Automation tool) is the manual pause/resume, while the Alpha5 tool has multi-core support (although time isn't exactly halved in my dualcore, but it does work faster) and the file skipping support.
User avatar
Laura Tempel
 
Posts: 3484
Joined: Wed Oct 04, 2006 4:53 pm

Post » Sun Apr 25, 2010 3:57 pm

You're welcome.

Actually, my preference was based on a small test last night. Basically, I thought that the latest version of PyFFI already supported multiple cores (read that somewhere). Hence, having the automater support it would not make a difference.

Further investigation though confirmed that this automater was making PyFFI only use 1 core. I just installed the other automater linked in the OP and I can confirm that both of my cores are now being used to PyFFI meshes (actually, the program does this by opening 2 copies of PyFFI at the same time). The last time I installed it though, my PC experienced a massive slowdown when running the automater, to the point that even my Win 7 "Ribbons" screensaver became a slideshow. That problem doesn't seem to be happening as I type this though so I don't know what happened. I just had a bad experience.

I would say that the is really the best PyFFI automater you can get. The only thing is that the INI needs some configuration before use.
First, put a larger number next to size_test. 10 should be adequate. Basically, this means that the optimised mesh must have been reduced in size to this percentage of the original or the optimised NIF is not saved (10 means the the size of the optimised NIF must be less than 1000% of the original). The problem though is that PyFFI optimises for performance, not small size. This sometimes causes the NIF to actually become bigger. Hence, a size test is not a good way of measuring PyFFIs effect on an NIF.
Second, delete everything under skip_list. PyFFI now automatically checks for NIFs that it shouldn't optimise. IF you don't feel completely comfortable though, you can leave "hair" in and delete everything else.
User avatar
Fiori Pra
 
Posts: 3446
Joined: Thu Mar 15, 2007 12:30 pm

PreviousNext

Return to IV - Oblivion