[info][relz] PyFFI - Python File Format Interface

Post » Thu May 19, 2011 8:09 pm

When I optimize via the .\in and .\out folders, some of the optimized nifs are bigger than the original. What do people usually do then.. replace with the optimized or keep the original?

They are bigger because of added fixes. That is normal. Use the optimized meshes.
User avatar
Roisan Sweeney
 
Posts: 3462
Joined: Sun Aug 13, 2006 8:28 pm

Post » Thu May 19, 2011 6:49 pm

I posted this on sourceforge a few days ago and have received a response.
I tried to substitute some paths with modify_substitutetexturepath.

I used this -a /.*\\textures\\(.*)/textures\\$1

I expected the $1 to be replaced with the string found with (.*) .. but it didn't.. does it not support this or does it need another format?

Anyone here know the answer?
User avatar
Minako
 
Posts: 3379
Joined: Sun Mar 18, 2007 9:50 pm

Post » Thu May 19, 2011 2:34 pm

I must be doing something wrong then because I added this:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" opt_collisiongeometry --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize.ini" --dest-dir= --source-dir= --pause --overwrite "%1"

based on the fact that the "Optimize with PyFFI" shell extension looks like this:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize.ini" --dest-dir= --source-dir= --pause --overwrite "%1"

and all I keep getting is an error telling me it failed the test on anything I try and use it on.

Is it not possible to have this option behave like the normal one? I don't really want to go scattering junk through my Program Files folders just to do this.


I had to use the --dest-dir= --source-dir= manual overrides because theres no optimize.ini for running the collision spell, and I don't know enough about pyffi to write one, the original command for regular optimize uses a dual ini entry, and default.ini just sets up the default runtime environment.

As for scattering junk in program files, doesn't add much more then using the /in and /out folders in program files, but you can always set your work working dir.
User avatar
Portions
 
Posts: 3499
Joined: Thu Jun 14, 2007 1:47 am

Post » Thu May 19, 2011 7:31 pm

Regarding the EGM files, it it just the EGM files themselves that are on the list of files not to optimize, or does it also apply to the assciated NIF files?

Never mind, I guess this from the Read Me means both the EGM and associated NIF should should not be PyFFI'ed:
"delete any files that have an egm (which would be files in a folder that are in the nature of helmet.nif and another by the name of helmet.egm), most open helmets and hair have egms."
User avatar
Anna Kyselova
 
Posts: 3431
Joined: Sun Apr 01, 2007 9:42 am

Post » Thu May 19, 2011 8:38 pm

Regarding the EGM files, it it just the EGM files themselves that are on the list of files not to optimize, or does it also apply to the assciated NIF files?

Never mind, I guess this from the Read Me means both the EGM and associated NIF should should not be PyFFI'ed:
"delete any files that have an egm (which would be files in a folder that are in the nature of helmet.nif and another by the name of helmet.egm), most open helmets and hair have egms."

PyFFI skips those (NIF files associated with EGMs) automatically (or it is supposed to.) Just check the "out" folder briefly to be sure.
User avatar
Lew.p
 
Posts: 3430
Joined: Thu Jun 07, 2007 5:31 pm

Post » Thu May 19, 2011 1:02 pm

When I was reading up on Pyffi the other day I saw a handy list of mods that already had Pyffi'd meshes (Better Cities was on there) - I can't find it now though.

Specifically wondering if it's worth pyffiing the stuff in UOP and QTP Redimized (is it already optimized?).
User avatar
Suzy Santana
 
Posts: 3572
Joined: Fri Aug 10, 2007 12:02 am

Post » Thu May 19, 2011 5:10 pm

When I was reading up on Pyffi the other day I saw a handy list of mods that already had Pyffi'd meshes (Better Cities was on there) - I can't find it now though.

Specifically wondering if it's worth pyffiing the stuff in UOP and QTP Redimized (is it already optimized?).

Depends on how much extra work you will accept. As you say they are optimized already but I usually run it anyway if I find the time since I am using other optimization spells and am somewhat more reckless in pushing the meshes around - that means I have to check a bit harder for faulty meshes. So I only do it with meshes when I got time.

Optimizing the collision meshes seems to always be an idea though.
Here is an example of Better Cities - Bravil Docks that has optimized it's Havok collision.
As you can see it does still optimize the collision if using the spell = opt_collisiongeometry
Mopper uses havok. Copyright 1999-2008 Havok.com Inc. (and its Licensors).All Rights Reserved. See www.havok.com for details.pyffi.toaster:3272:INFO:  --- opt_collisiongeometry ---pyffi.toaster:3272:INFO:    ~~~ NiNode [AnvilDockEnd01] ~~~pyffi.toaster:3272:INFO:      ~~~ bhkCollisionObject [] ~~~pyffi.toaster:3272:INFO:        ~~~ bhkRigidBody [] ~~~pyffi.toaster:3272:INFO:          ~~~ bhkMoppBvTreeShape [] ~~~pyffi.toaster:3272:INFO:            removing duplicate verticespyffi.toaster:3272:INFO:            (num vertices in collision shape was 74 and is now 48)pyffi.toaster:3272:INFO:            removing duplicate trianglespyffi.toaster:3272:INFO:            (num triangles in collision shape was 58 and  is now 58)pyffi.toaster:3936:INFO:  --- opt_collisiongeometry ---pyffi.toaster:3936:INFO:    ~~~ NiNode [AnvilBoardwalkCorner01] ~~~pyffi.toaster:3936:INFO:      ~~~ bhkCollisionObject [] ~~~pyffi.toaster:3936:INFO:        ~~~ bhkRigidBodyT [] ~~~pyffi.toaster:2504:INFO:  --- opt_collisiongeometry ---pyffi.toaster:3936:INFO:          ~~~ bhkMoppBvTreeShape [] ~~~pyffi.toaster:2504:INFO:    ~~~ NiNode [AnvilDock02] ~~~pyffi.toaster:3936:INFO:            removing duplicate verticespyffi.toaster:2504:INFO:      ~~~ bhkCollisionObject [] ~~~pyffi.toaster:2504:INFO:        ~~~ bhkRigidBody [] ~~~pyffi.toaster:3936:INFO:            (num vertices in collision shape was 48 and is now 26)pyffi.toaster:2504:INFO:          ~~~ bhkMoppBvTreeShape [] ~~~pyffi.toaster:2504:INFO:            removing duplicate verticespyffi.toaster:3936:INFO:            removing duplicate trianglespyffi.toaster:3936:INFO:            (num triangles in collision shape was 22 and is now 22)pyffi.toaster:2504:INFO:            (num vertices in collision shape was 122 and is now 64)pyffi.toaster:2504:INFO:            removing duplicate trianglespyffi.toaster:2504:INFO:            (num triangles in collision shape was 82 and is now 82)pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 0] ~~~pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 1] ~~~pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 2] ~~~pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 3] ~~~pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 4] ~~~pyffi.toaster:3184:INFO:        ~~~ NiTriShape [Tri AnvilBoardwalkStair02 5] ~~~pyffi.toaster:3184:INFO:  writing ./out\meshes\architecture\bettercitiesresources\docks\bravil\bravilboardwalkstair02.nifpyffi.toaster:3184:INFO:=== ./in\meshes\architecture\bettercitiesresources\docks\bravil\bravildockend01_far.nif ===pyffi.toaster:3184:INFO:=== ./in\meshes\architecture\bettercitiesresources\docks\bravil\bravildockstart01.nif ===


:foodndrink:
User avatar
Charles Mckinna
 
Posts: 3511
Joined: Mon Nov 12, 2007 6:51 am

Post » Thu May 19, 2011 11:31 am

I had to use the --dest-dir= --source-dir= manual overrides because theres no optimize.ini for running the collision spell, and I don't know enough about pyffi to write one, the original command for regular optimize uses a dual ini entry, and default.ini just sets up the default runtime environment.


How would one go about doing this? I am in the process of pyffing my meshes, and would love to also optimize my collision, but I'm very confused as to how to go about it. Is there maybe a guide to follow, or instructions?



Two more questions : ), when I repack my pyffied meshes back into the BSA, is there a recommended level of compression? I had read a post a while back (though I can't find it now) that says compressed BSAs take longer to load. Can anyone verify that? Also if the BSA is compressed it takes up less space on the hard drive, would the files loaded from the BSA also take up less process memory when running Oblivion? Just curious as I'm all for keeping the Oblivion process size down : ).
User avatar
Mandi Norton
 
Posts: 3451
Joined: Tue Jan 30, 2007 2:43 pm

Post » Thu May 19, 2011 4:44 pm

How would one go about doing this? I am in the process of pyffing my meshes, and would love to also optimize my collision, but I'm very confused as to how to go about it. Is there maybe a guide to follow, or instructions?



Two more questions : ), when I repack my pyffied meshes back into the BSA, is there a recommended level of compression? I had read a post a while back (though I can't find it now) that says compressed BSAs take longer to load. Can anyone verify that? Also if the BSA is compressed it takes up less space on the hard drive, would the files loaded from the BSA also take up less process memory when running Oblivion? Just curious as I'm all for keeping the Oblivion process size down : ).

Use 'No Compression'. It makes sense that it would take longer for the game to load compressed files because it has to uncompress the data before interpreting it. Since you are new to this, just try out the optimized meshes. You need to test those to make sure nothing went wrong. The opening post of my site's thread (second link in my sig) has BSA repacking instructions.
User avatar
Lalla Vu
 
Posts: 3411
Joined: Wed Jul 19, 2006 9:40 am

Post » Thu May 19, 2011 11:08 am

Use 'No Compression'. It makes sense that it would take longer for the game to load compressed files because it has to uncompress the data before interpreting it. Since you are new to this, just try out the optimized meshes. You need to test those to make sure nothing went wrong. The opening post of my site's thread (second link in my sig) has BSA repacking instructions.


Yeah that makes sense about the BSA compression. Could I expect to see any amount of performance increase if I took the textures BSA and uncompressed that also? Thanks for the help : )
User avatar
Trevor Bostwick
 
Posts: 3393
Joined: Tue Sep 25, 2007 10:51 am

Post » Thu May 19, 2011 3:09 pm

PyFFI skips those (NIF files associated with EGMs) automatically (or it is supposed to.) Just check the "out" folder briefly to be sure.


You're right! Excellent. :)
User avatar
Ashley Hill
 
Posts: 3516
Joined: Tue Jul 04, 2006 5:27 am

Post » Fri May 20, 2011 12:04 am

If I use the opt_collisiongeometry spell should the same files be skipped as with the normal optimize spell?
User avatar
Javaun Thompson
 
Posts: 3397
Joined: Fri Sep 21, 2007 10:28 am

Post » Thu May 19, 2011 8:06 am

If I use the opt_collisiongeometry spell should the same files be skipped as with the normal optimize spell?

I started with that but since it optimizes something different I started optimizing everything. Some of the problematic meshes that gets skipped have no collision either; there might be a wholly new skip list when using opt_collisiongeometry.

So far I have just discovered a possible problem with some planks , but it might have been a fluke, I just noticed it. With the new PyFFi I have also started to optimize much more and have come across less problems than earlier versions. Mark, I have used the traditional skipping when optimizing the original meshes in Oblivion - meshes.bsa just to be on the safe side for now.

I am going to redo the planks and see if my problem persists (right now I am knee deep in the planks, as if the mesh has lost a layer of collision). An other thing is that I can't get Duke Patrics Basic Hypothermia's rain protection to kick in - although I had a new install and it was a new version of Duke's so the problem might not be the optimizing of the collision meshes.

Not really noticed any problems - nothing that I have seen has lost its collision properties, people are not running through walls or disappearing through floors or some such.

I will look forward to you posting your results with this! As I said I did it on a new install and can thus not compare how much it helped. But overall my new game is smooth as butter when my old wasn't - and that had only optimized meshes, not the collision.

:foodndrink:
User avatar
Jani Eayon
 
Posts: 3435
Joined: Sun Mar 25, 2007 12:19 pm

Post » Thu May 19, 2011 12:55 pm

Hi Tomlong,

1) so with Pyffi 2.15 I can right-click entire Meshes folders (without manually moving out /moving back the Creatures and Characters ones), right?

2) can I use the right-click as well when optimizing BSAs, or following your method #2 is the only solution?

3) a file redater used on the finished (optimized) BSA brings back to 2006 the stamp of all contained files, so I don't have to worry about further archive invalidation etc, right?

3) can you please specify what you mean with "leading spaces?", and how copy/moving files can create problems before repacking?
(sorry english isn't my first language - I'm referring to "The first key step is to repack the BSA without moving or copying the optimized files; otherwise, the leading spaces would be truncated.")

Thanks :-)
User avatar
Vicky Keeler
 
Posts: 3427
Joined: Wed Aug 23, 2006 3:03 am

Post » Fri May 20, 2011 12:58 am

Hi Tomlong,

1) so with Pyffi 2.15 I can right-click entire Meshes folders (without manually moving out /moving back the Creatures and Characters ones), right?

2) can I use the right-click as well when optimizing BSAs, or following your method #2 is the only solution?

3) a file redater used on the finished (optimized) BSA brings back to 2006 the stamp of all contained files, so I don't have to worry about further archive invalidation etc, right?

3) can you please specify what you mean with "leading spaces?", and how copy/moving files can create problems before repacking?
(sorry english isn't my first language - I'm referring to "The first key step is to repack the BSA without moving or copying the optimized files; otherwise, the leading spaces would be truncated.")

Thanks :-)

1) Yes.
2) I just posted instructions for users using the in/out folder method...
4) ...because if you use the right-click method, you do not have to worry about the leading spaces as you never copy/move the files anywhere
3) A file redater allows you to set the BSA times to any date. OBMM's reset BSA timestamps function resets the dates to 2006. It would be ideal if no mods' BSAs overwrote vanilla files, but that is not the case, so resetting all BSAs to 2006 may cause problems. Setting the originals back to 2006 is fine, however.
4) Leading spaces means spaces at the front of the filename. If you look at the filenames in the codebox, you can see them there. The problem is that leading spaces are not normally supported, and copying/moving the files will truncate/remove them. For example, if you try to rename a file " food", the resultant file in Windows would have the name "food". If you use the context menu (right-click) method, and pack the BSA without moving the extracted files, you do not have to worry about this.
User avatar
emma sweeney
 
Posts: 3396
Joined: Fri Sep 22, 2006 7:02 pm

Post » Thu May 19, 2011 11:12 pm

Hmm....using the Pyffi Script version alpha 5 aka Pyffi Automation Utility with the latest version of Pyffi leads to most of the files being skipped.

Using the right click om meshes folder method I notice that files with the word 'helmet' are skipped, like melushelmetaltprop.nif and melushelmetprop.nif, which are not helmets with an associated EGM file, but files located in the OOO Architecture folder.

Are all helmets unsafe to process or only those with EGM files?
User avatar
Prohibited
 
Posts: 3293
Joined: Tue Jun 12, 2007 6:13 am

Post » Thu May 19, 2011 11:14 am

Hmm....using the Pyffi Script version alpha 5 aka Pyffi Automation Utility with the latest version of Pyffi leads to most of the files being skipped.

Using the right click om meshes folder method I notice that files with the word 'helmet' are skipped, like melushelmetaltprop.nif and melushelmetprop.nif, which are not helmets with an associated EGM file, but files located in the OOO Architecture folder.

Are all helmets unsafe to process or only those with EGM files?

PyFFI not only skips EGMk-associated NIFS, but also it skips file with names containing strings specified in the PyFFI optimize INI.
User avatar
Matthew Warren
 
Posts: 3463
Joined: Fri Oct 19, 2007 11:37 pm

Post » Thu May 19, 2011 5:46 pm

So I just optimized all the vanilla, DLC, and QTP3R (in separate folders in .\in) at the same time.... I then overwrote the .\out with my .\in folder :banghead:
User avatar
Dalley hussain
 
Posts: 3480
Joined: Sun Jun 18, 2006 2:45 am

Post » Thu May 19, 2011 1:02 pm

1) Yes.
2) I just posted instructions for users using the in/out folder method...
4) ...because if you use the right-click method, you do not have to worry about the leading spaces as you never copy/move the files anywhere
3) A file redater allows you to set the BSA times to any date. OBMM's reset BSA timestamps function resets the dates to 2006. It would be ideal if no mods' BSAs overwrote vanilla files, but that is not the case, so resetting all BSAs to 2006 may cause problems. Setting the originals back to 2006 is fine, however.
4) Leading spaces means spaces at the front of the filename. If you look at the filenames in the codebox, you can see them there. The problem is that leading spaces are not normally supported, and copying/moving the files will truncate/remove them. For example, if you try to rename a file " food", the resultant file in Windows would have the name "food". If you use the context menu (right-click) method, and pack the BSA without moving the extracted files, you do not have to worry about this.


Thanks again Tomlong for your kind reply.

I've been researching Pyffi for a few hours, and I'm still wondering

1. On a clean install of Oblivion, should I optimize the Data/Meshes folder as well, or just the BSA-packed one?

2. Does geometry optimization happen automatically when right-clicking a folder with 2.1.5 release?

3. If not, do you think (for a gamer like me) that would it be wortwhile/safe enough to go for it via command-line (if so, anyone to suggest?)
or "right-clicking folder" should be all I do?

Cheers!
User avatar
Sophie Payne
 
Posts: 3377
Joined: Thu Dec 07, 2006 6:49 am

Post » Fri May 20, 2011 1:44 am


1. On a clean install of Oblivion, should I optimize the Data/Meshes folder as well, or just the BSA-packed one?



On a clean install of Oblivion, there are no Data/Meshes folder. B)
User avatar
Heather M
 
Posts: 3487
Joined: Mon Aug 27, 2007 5:40 am

Post » Fri May 20, 2011 2:09 am

Perhaps someone could explain this cryptic tidbit of information for me?

Further optimization with PyFFI for _FAR NIF files ONLY! --Vorians
"C:\Program Files\Python26\python.exe" "C:\Program Files\Python26\Scripts\niftoaster.py" fix_deltangentspace Meshes


I assume it's something typed into command prompt, but I don't know what directory I should be in when I do it, nor do I know if it will optimize the _FAR nif files on its own or if I should make sure everything but the _FAR nif files are not where it's pointing to.

Also, I couldn't find any instructions on optimizing the Shivering Isles/Knights meshes, or any of the DLC meshes. I assume at the very least I want to decompress all my bsa's and then recreate them with no compression though.
User avatar
jess hughes
 
Posts: 3382
Joined: Tue Oct 24, 2006 8:10 pm

Post » Fri May 20, 2011 2:18 am

On a clean install of Oblivion, there are no Data/Meshes folder. B)


Thanks PetrusOctavianus, I've been intensely studying Oblivion modding for 3 weeks now but at times I'm still unprepared on the basics lol
User avatar
Katy Hogben
 
Posts: 3457
Joined: Mon Oct 30, 2006 12:20 am

Post » Thu May 19, 2011 1:06 pm

A question:

pyffi.toaster:2776:INFO:          ~~~ bhkMoppBvTreeShape [] ~~~pyffi.toaster:2776:INFO:            removing duplicate verticespyffi.toaster:2776:INFO:            (num vertices in collision shape was 206 and is now 104)pyffi.toaster:2776:INFO:            removing duplicate trianglespyffi.toaster:2776:INFO:            (num triangles in collision shape was 136 and is now 136)


This has been the typical output of meshes I've been sampling for testing the collision opt. It only ever seems to remove verts, so far it has never reduced the number of triangles. Is this normal? Or have I just not run into a mesh so far where it has actual duplicated triangles?
User avatar
alicia hillier
 
Posts: 3387
Joined: Tue Feb 06, 2007 2:57 am

Post » Thu May 19, 2011 3:16 pm

A question:

pyffi.toaster:2776:INFO:          ~~~ bhkMoppBvTreeShape [] ~~~pyffi.toaster:2776:INFO:            removing duplicate verticespyffi.toaster:2776:INFO:            (num vertices in collision shape was 206 and is now 104)pyffi.toaster:2776:INFO:            removing duplicate trianglespyffi.toaster:2776:INFO:            (num triangles in collision shape was 136 and is now 136)


This has been the typical output of meshes I've been sampling for testing the collision opt. It only ever seems to remove verts, so far it has never reduced the number of triangles. Is this normal? Or have I just not run into a mesh so far where it has actual duplicated triangles?

I have just collision opt. B&M, I searched the log I made and found 3 instances of it... so it does happen :)
User avatar
Khamaji Taylor
 
Posts: 3437
Joined: Sun Jul 29, 2007 6:15 am

Post » Thu May 19, 2011 7:45 pm

Ok, so it's just somewhat rare then. I can handle that.

I don't suppose there's some nifty way to get the "right click on a folder" method to spit out a log file is there? If not I can just write a batch file to do the work instead.
User avatar
Darrell Fawcett
 
Posts: 3336
Joined: Tue May 22, 2007 12:16 am

PreviousNext

Return to IV - Oblivion