[RELZWIPZ]PyFFI: Python File Format Interface

Post » Tue May 08, 2012 9:22 pm

@Reigor: The actual nif files. Just a few of the ones that report errors should be enough.
User avatar
Anthony Diaz
 
Posts: 3474
Joined: Thu Aug 09, 2007 11:24 pm

Post » Tue May 08, 2012 3:16 pm

http://www.gamesas.com/index.php?/topic/1172029-relz-oblivion-pyffi-optimization-kit/page__view__findpost__p__18913119 about which PyFFI approach to use as Ulrim's kit looked interesting.

I couldn't get it to work with Python 27 and was directed back here and told this now also contained his four step process. I've used this method before (way back with version 1.4 I think) so I know about the Toaster/in and out folders.

Question though - will running this as per the opening post here:
2. Place the meshes into your PyFFI’s in folder (ex: C:\Program Files\PyFFI\utilities\toaster\in) and run PyFFI from the oblivion_optimize.ini
run all four passes. Or do I use the bat file?

Seems the opening post is not up to date on which thingy to click on.

thanks
User avatar
Ricky Meehan
 
Posts: 3364
Joined: Wed Jun 27, 2007 5:42 pm

Post » Wed May 09, 2012 3:37 am

Follow up question as after doing the above and seeing that it did do all 4 passes.

I realized that I forgot about the damn leading spaces issue on cave exteriors ... again. I hate that.

So now I'm going about it this way - I extracted the meshes BSA then using the right click optimize method which brings up the cmd shell I'm used to. The idea being that that I will then repackage from the replaced files in the same spot.

Will that solve the leading spaces issue?
Does doing it this way (right click method) render all 4 passes? If not then what do I do?

Is there a value in optimizing the _far nifs as per this section:
Further optimization for _far.NIFs (VWD meshes) can be called with the following batch script:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive opt_cleanfarnif --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"
I don't see that as part of the four passes - is it something I must do separately?

thanks for help - awaiting answers.
User avatar
Jerry Jr. Ortiz
 
Posts: 3457
Joined: Fri Nov 23, 2007 12:39 pm

Post » Tue May 08, 2012 10:09 pm

If you're using PyFFI 2.1.10 (and you should be) it's been reduced to a 3-step process.

Drop the files into C:\Program Files\PyFFI\utilities\toaster\in
Run the batch file in C:\Program Files\PyFFI\utilities\toaster
Wait.....
When done, copy the files from the C:\Program Files\PyFFI\utilities\toaster\out folder over the ones in C:\Program Files\PyFFI\utilities\toaster\in
Repack as desired from there.

On the vanilla meshes, expect this 3-stage process to take ~4 hours on a good rig.
User avatar
Janeth Valenzuela Castelo
 
Posts: 3411
Joined: Wed Jun 21, 2006 3:03 am

Post » Wed May 09, 2012 4:02 am

That is the way I've done it before.

The problem is if I move those cave entrance meshes at all then the leading space gets truncated and will show as missing meshes in game. Whether I copy the out folder over the in folder and replace all or copy the in folder over the out folder and don't replace all (only move what was not optimized) then that means moving those files and truncating the names again.

If I extract the meshes bsa to the in folder and run as you describe then will those leading spaces be truncated in the out folder?

And about the _far nifs - is that part of the process now automatically or must I do that separately?

thanks
User avatar
Brian LeHury
 
Posts: 3416
Joined: Tue May 22, 2007 6:54 am

Post » Wed May 09, 2012 3:24 am

Use the UOP Supplemental. The truncated filenames won't matter then.

_far.nif cleanup and optimization has been included into the 3-stage process.
User avatar
Jamie Moysey
 
Posts: 3452
Joined: Sun May 13, 2007 6:31 am

Post » Tue May 08, 2012 5:33 pm

hi
Is there any way to set Python path pyffi uses ? I have two Python installations and want pyffi to work with 26.5 instead 27.1.
I have crashes while CUO_Bruma meshes are proceeded, but the meshes look OK in nifscope, so I would like to check out under other Python version.

edited :
to make things clear. I can pyffi meshes one by one. I cannot do it for all at once. This CUO_Bruma have several meshes only. CUO_res has about 180 mega of meshes - no problems.

edited2:
Directly crashes this lite spamer mopper .

edited3:
win7/64 pyffi 2.1.10

edited4:
right click method only. by ini - OK.
User avatar
sharon
 
Posts: 3449
Joined: Wed Nov 22, 2006 4:59 am

Post » Tue May 08, 2012 4:23 pm

Python version shouldn't matter much. I'm doing all development on Python 2.7 so that's a safe bet (for the "old" version - in fact I'll be switching to Python 3.2 for the next release).

If you want to change the interpreter used by PyFFI (right-click), you have to do it either via regedit (this is a bit tricky), or uninstall python 2.7 and reinstall pyffi.

If you use the batch file, then you can just change it in your favorite editor.
User avatar
Amysaurusrex
 
Posts: 3432
Joined: Wed Aug 09, 2006 2:45 pm

Post » Tue May 08, 2012 12:12 pm

Python version shouldn't matter much. I'm doing all development on Python 2.7 so that's a safe bet (for the "old" version - in fact I'll be switching to Python 3.2 for the next release).

If you want to change the interpreter used by PyFFI (right-click), you have to do it either via regedit (this is a bit tricky), or uninstall python 2.7 and reinstall pyffi.

If you use the batch file, then you can just change it in your favorite editor.

Thank you. I use 2.7 for Wry Bash. In my opinion problem is how 2.7 works on win7/64. Working in Bain I may notice terrible lags just after I call another process (and kill them ) example - Word. Python after that operation uses only one core and about 600 mg ram only.
User avatar
Mistress trades Melissa
 
Posts: 3464
Joined: Mon Jun 19, 2006 9:28 pm

Post » Tue May 08, 2012 3:24 pm

Is there any benefit to running Tes4LODGEN with newly pyffied meshes?
User avatar
pinar
 
Posts: 3453
Joined: Thu Apr 19, 2007 1:35 pm

Post » Wed May 09, 2012 4:32 am

Just a FYI. Regarding the AQSpout.nif that is being skipped, it is from Let the People Drink (Meshes\Architecture\ImperialCity\Aquaducts + Fountains\AQSpout.nif). It causes the mopper.exe to crash. A working AQSpout.nif can be found in the "Better Cities - Meshes.bsa" (meshes\archithecture\bettercitiesresources\ltpd\aqspout.nif) from "Better Cities Resources 5.0.0".
User avatar
Janine Rose
 
Posts: 3428
Joined: Wed Feb 14, 2007 6:59 pm

Post » Tue May 08, 2012 11:44 pm

Is there any benefit to running Tes4LODGEN with newly pyffied meshes?
It shouldn't matter unless you altered your load order after running PyFFI.
User avatar
Wane Peters
 
Posts: 3359
Joined: Tue Jul 31, 2007 9:34 pm

Post » Tue May 08, 2012 10:25 pm

Um where do I begin... I was reading through this thread and then the Oblivion wiki to become familiar with pyffi process. Thought I was doing it the right way until I read something in this thread. I've been using the pyffi option by right clicking on the file folder and choose optimize with pyffi. I don't know if this is going to cause abnormalities because I was optimizing the armor folder for all of replacers using this method. What is the difference between this method and using the in folder from the pyffi directory and initializing pyffi with the batch file? Did I unknowingly mess up the meshes instead of optimizing them? :banghead: :facepalm: Is there an option that can reverse the process so I can do it the right way? :nope: :ahhh:
User avatar
Wayne Cole
 
Posts: 3369
Joined: Sat May 26, 2007 5:22 am

Post » Tue May 08, 2012 11:12 pm

I'm having a strange issue..after pyffing qtp3 redimized meshes,I cannot load a savegame because I get a full black screen after the game has been loaded.Also,If I load a different save and go to chorrol,after a min or so I get a CTD.I get the same issue with Nifopt.What's going on?I've reinstalled the game before pyffing the meshes.

(video card: msi gtx 560 ti 1gb ddr5)
User avatar
quinnnn
 
Posts: 3503
Joined: Sat Mar 03, 2007 1:11 pm

Post » Tue May 08, 2012 5:18 pm

For info, looks like 2.1.11 was released on 26th November. https://github.com/amorilia/pyffi/blob/python25/CHANGELOG.rst. http://sourceforge.net/projects/pyffi/files/pyffi/2.1.11/.

There's also a "pyffi-py3k" version, but I'm failing to find information on how that differs...
User avatar
Kevan Olson
 
Posts: 3402
Joined: Tue Oct 16, 2007 1:09 am

Post » Wed May 09, 2012 3:45 am

Py3K (of which PyFFI 2.2.0 is the first in the series) uses Python 3.x instead of Python 2.x. I haven't tried it yet - I'm sticking to PyFFI 2.1.11 plus Python 2.7.3 - but others have reported success with this new combination.

Why couldn't they have just called it PyFFI 3.0 to make things simple?
User avatar
Tarka
 
Posts: 3430
Joined: Sun Jun 10, 2007 9:22 pm

Post » Tue May 08, 2012 5:05 pm

Um where do I begin... I was reading through this thread and then the Oblivion wiki to become familiar with pyffi process. Thought I was doing it the right way until I read something in this thread. I've been using the pyffi option by right clicking on the file folder and choose optimize with pyffi. I don't know if this is going to cause abnormalities because I was optimizing the armor folder for all of replacers using this method. What is the difference between this method and using the in folder from the pyffi directory and initializing pyffi with the batch file? Did I unknowingly mess up the meshes instead of optimizing them? [img]http://www.gamesas.com/images/smilie/banghead.gif[/img] [img]http://www.gamesas.com/images/smilie/facepalm.gif[/img] Is there an option that can reverse the process so I can do it the right way? [img]http://www.gamesas.com/images/smilie/nope.gif[/img] [img]http://www.gamesas.com/images/smilie/ahhh.gif[/img]
You're fine - I do it that way all the time, and I've never had any problems.
User avatar
Project
 
Posts: 3490
Joined: Fri May 04, 2007 7:58 am

Post » Tue May 08, 2012 8:03 pm

Python 3.0 will become more relevant once people start needing Blender for Skyrim since the newer versions of Blender also require it. So PyFFI has to follow suit before there can be new nifscripts for import/export. Fortunately Bash also has a Python 3.0 version in the works.
User avatar
Skrapp Stephens
 
Posts: 3350
Joined: Mon Aug 06, 2007 5:04 am

Post » Tue May 08, 2012 8:39 pm

Trying to track down a consistent crash-to-desktop issue that I'm getting with my new installation. PyFFI is one of the possible culprits at this time.
Right now I am loading NOTHING other than Oblivion.esm, DLCShiveringIsles.esp, and the unofficial patches, and I get a crash-to-desktop every time I exit the Tutorial Dungeon or use the console to go to any random outdoor location. This is after I have run all the vanilla BSAs through PyFFI, combined together the Oblivion and DLCShiveringIsles BSAs, added in the relevant Unofficial Patch files, and then used BSAOpt to recreate the BSA.

The crashing is definitely due to Oblivion - Meshes.bsa, because when I replace it with the vanilla copy, the crashing stops. So, one (or more) of these four things must be the culprit:
  • PyFFI. I'm using Python 2.7.2 x86 with PyFFI 2.1.11 and the PyFFI Optimization Kit.
  • BSAOpt. It's possible that something is going wrong when I recreate the BSAs. But, unzipping and rezipping the vanilla BSAs, without otherwise altering them, causes no crash.
  • Adding the DLCShiveringIsles BSAs into the Oblivion BSAs. The crash is NOT due to the INI file being set up incorrectly - it it was then Oblvion wouldn't load at all. Maybe the game is hard coded to expect the DLCShiveringIsles BSAs to exist?
  • Adding the Unofficial Patch files into the Oblivion BSAs.

I'll continue with my testing to figure out which of the above is the root cause. Does anyone else have prior experience (or advice) that could help me?
User avatar
Gwen
 
Posts: 3367
Joined: Sun Apr 01, 2007 3:34 am

Post » Tue May 08, 2012 1:59 pm

PyFFI comes with its own internal optimization kit, so you shoudln't be using one found elsewhere. Did you use the 3-stage process for this?

How large is the mesh BSA file now?
User avatar
Victoria Bartel
 
Posts: 3325
Joined: Tue Apr 10, 2007 10:20 am

Post » Tue May 08, 2012 10:16 pm

Maybe the game is hard coded to expect the DLCShiveringIsles BSAs to exist?
Maybe it is indeed.

Otherwise, as Arthmoor already suggests, a BSA file must not exceed a file size of 2GB. Maybe with all the integration you did, you went beyond that?
User avatar
latrina
 
Posts: 3440
Joined: Mon Aug 20, 2007 4:31 pm

Post » Tue May 08, 2012 6:29 pm

Maybe the game is hard coded to expect the DLCShiveringIsles BSAs to exist?
Nope, I can confirm that this isn't the case in my game. Unpacked all my BSAs and using BAIN to install the resources with 7z packages (except the Voices BSAs)

Replaced all Oblivion BSAs with renamed ArchiveInvalidation.bsas (from Wrye Bash) and combine Oblivion - Voices2 into Oblivion - Voices1, Renamed DLCShiverngIsles - Voices to Oblivion - Voices2. Made sure to repack them using 0 compression in BSAOpt.

Result, I only have Oblivion BSAs (of which only the Voices ones actually contain resources, Voice2 containing SI voices)
User avatar
Richard
 
Posts: 3371
Joined: Sat Oct 13, 2007 2:50 pm

Post » Tue May 08, 2012 12:39 pm

PyFFI comes with its own internal optimization kit, so you shoudln't be using one found elsewhere. Did you use the 3-stage process for this?
How large is the mesh BSA file now?
Here's what I've been using: http://www.tesnexus.com/downloads/file.php?id=37463 (It's a 4 stage process created by Ulrim.) From what I can tell, it's similar to PyFFI's own version but with a few tweaks.

Otherwise, as Arthmoor already suggests, a BSA file must not exceed a file size of 2GB. Maybe with all the integration you did, you went beyond that?
Nope, none of my BSAs are over 2GB (though Textures comes fairly close).

Nope, I can confirm that this isn't the case in my game. Unpacked all my BSAs and using BAIN to install the resources with 7z packages (except the Voices BSAs)

Replaced all Oblivion BSAs with renamed ArchiveInvalidation.bsas (from Wrye Bash) and combine Oblivion - Voices2 into Oblivion - Voices1, Renamed DLCShiverngIsles - Voices to Oblivion - Voices2. Made sure to repack them using 0 compression in BSAOpt.

Result, I only have Oblivion BSAs (of which only the Voices ones actually contain resources, Voice2 containing SI voices)
Hmm...if that's the case, then either PyFFI (with the Kit I'm using) or BSAOpt are to blame...
User avatar
Arnold Wet
 
Posts: 3353
Joined: Fri Jul 07, 2006 10:32 am

Post » Tue May 08, 2012 9:20 pm

Here's what I've been using: http://www.tesnexus.com/downloads/file.php?id=37463 (It's a 4 stage process created by Ulrim.) From what I can tell, it's similar to PyFFI's own version but with a few tweaks.
That's older than PyFFI's internal process :P (also, if you search for its thread here in gamesas, you'll find that it's buggy and has been preceded by a 5-step process whose link is in that thread) but none of that matters because PyFFI's internal process is the latest and most bug-free one.

I personally still use the no-skiplist shell_optimize.ini but the safer alternative would ofcourse be the .bat files in PyFFI :)
User avatar
Milagros Osorio
 
Posts: 3426
Joined: Fri Aug 25, 2006 4:33 pm

Post » Tue May 08, 2012 8:24 pm

That's older than PyFFI's internal process :tongue: (also, if you search for its thread here in gamesas, you'll find that it's buggy and has been preceded by a 5-step process whose link is in that thread) but none of that matters because PyFFI's internal process is the latest and most bug-free one.

I personally still use the no-skiplist shell_optimize.ini but the safer alternative would ofcourse be the .bat files in PyFFI :smile:
Good to know! I was under the impression that the reverse was true - that the Kit was newer than the internal process. That's what I get for not reading the documentation carefully.


It looks like faulty PyFFI'ing is the reason for my crashing. I created separate Oblivion - Meshes and Shivering Isles - Meshes BSAs to be on the safe side; the only change I made to the BSAs (besides adding in the Unofficial Patch files) was to PyFFI them. Then I just zipped them up - no compression. Insta-crash. Guess I'll try it again!
User avatar
Everardo Montano
 
Posts: 3373
Joined: Mon Dec 03, 2007 4:23 am

PreviousNext

Return to IV - Oblivion