[INFO] [RELZ]PyFFI - Python File Format Interface

Post » Mon Apr 26, 2010 1:54 am

Then it's not working right. I meant to report this sooner, but with 2.1.2 it did NOT ignore a mesh that has an egm file with it. It ended up getting optimized anyway. If it helps, all I did was right-click the folder and told PyFFI to optimize it. Then ended up seeing the helmet mesh get some verts chopped off. No idea if it's messed up though.

hmmm rats... I can't duplicate that; it should do everything except for the opt_geometry spell (and does for me)... It checks to see if there is a file named %filename-extension%.egm in the folder that it is in... anything else and neither the game nor PyFFI will see it... could you double check that?
What is the latest greatest...umm stable/reliable version I should be using for my meshes?

I am using Pyffi 2.0.5
and Blender 2.4.9b

PyFFI 2.1.2 (full release) or 2.1.3 (dev release; I've yet to see any errors but I haven't been in game much the last couple weeks so... but code wise very stable... also some significant speed improvements)
Blender 2.49b
and Blender NifScripts 2.5.1 (or in-dev versio nof 2.5.2)
Pacific Morrowind
User avatar
Tinkerbells
 
Posts: 3432
Joined: Sat Jun 24, 2006 10:22 pm

Post » Sun Apr 25, 2010 8:10 pm

  • right click on oblivion_optimize.ini and select "Run with PyFFI"


This doesn't appear as an option for me when I right click on oblivion_optimize.ini. Did I mess up the installation in some way?
User avatar
Stefanny Cardona
 
Posts: 3352
Joined: Tue Dec 19, 2006 8:08 pm

Post » Sun Apr 25, 2010 6:19 pm

The latest version is 2.1.3, which can be http://www.sendspace.com/file/jtljr6. You can find more info about the 2.1.3 release http://www.gamesas.com/bgsforums/index.php?s=&showtopic=1061938&view=findpost&p=15619151. The latest stable one though is either 2.1.3 or 2.1.2. Take your pick.

PyFFI 2.1.2 (full release) or 2.1.3 (dev release; I've yet to see any errors but I haven't been in game much the last couple weeks so... but code wise very stable... also some significant speed improvements)
Blender 2.49b
and Blender NifScripts 2.5.1 (or in-dev versio nof 2.5.2)

EXCELLENT! Thanks. Seems to be working fine.
Blender 2.4.9b
Pyffi 2.1.3
NifScripts 2.5.1

One more question...
I have been making some detailed (bigger) meshes in blender and have went through them very thoroughly by hand in the making of them.
When I export a nif in blender lets say it comes out to 1 MB, OK... Now if I run Pyffi on the mesh again after export, it will add just a little bit of file size, say 1.06MB for example. Is this actually increasing performance, since I already know there are no dup verts or such. Did blender do a better job the first time or should I right click on the mesh after export and optimise again, thus increasing the file size just a bit?

Metallicow
User avatar
Bedford White
 
Posts: 3307
Joined: Tue Jun 12, 2007 2:09 am

Post » Sun Apr 25, 2010 11:58 pm

@Frodo
I think it is supposed to say "right click on the mesh folder and select "Run with PyFFI"

@Metallicow
PyFFI optimises for performance, not file size. Sometimes, the pursuit for performance causes the mesh to increase in size. One example is the relationship between NiTriStrips and NiTriShapes.

Basically, Strips take up less space (correct me if I'm wrong) but if the strip length is less than 10, changing a Shape into a Strip causes an increase in stutter and reduces game performance. Hence, any Strips with a length of less than 10 are turned into Shapes and any Shapes that would create a Strip with a length of less than 10 remain a Shape, even if this means that the mesh size increases.
User avatar
Hayley O'Gara
 
Posts: 3465
Joined: Wed Nov 22, 2006 2:53 am

Post » Sun Apr 25, 2010 3:47 pm

@Metallicow
PyFFI optimises for performance, not file size. Sometimes, the pursuit for performance causes the mesh to increase in size. One example is the relationship between NiTriStrips and NiTriShapes.

Basically, Strips take up less space (correct me if I'm wrong) but if the strip length is less than 10, changing a Shape into a Strip causes an increase in stutter and reduces game performance. Hence, any Strips with a length of less than 10 are turned into Shapes and any Shapes that would create a Strip with a length of less than 10 remain a Shape, even if this means that the mesh size increases.


Yes, it's pretty much that - more precisely, pyffi looks at a *weighed average* of the strip length to decide between strip or trishape. The heuristic was determined by experimentation and checking framerates in-game. If anyone finds any cases where you can do substantially better than the current heuristic, feel free to report.

@Arthmoor: As Pacific Morrowind already asked, can you confirm that .egm related nifs are skipped? (I also checked again and it seems to work as intended...)

@FrodoTheDarkLord: Just to confirm that I really intended to make this work on .ini files to enable customization of the options. Are you using the latest dev version (linked above in my earlier post)? It won't be available with any other version.

[EDIT] @Frodo: Silly me! Of course if you see the ini file, you should be using the latest version :) I'll check this again on Vista and XP, maybe I missed something in the install script.
User avatar
Miragel Ginza
 
Posts: 3502
Joined: Thu Dec 21, 2006 6:19 am

Post » Sun Apr 25, 2010 2:05 pm

I'd like to also comment about running PyFFi on .egm associated meshes with v2.1.2
I also used the right click option to optimize using PyFFI and had some negative results.
Particularly on a mod called XEO5 which has many hair and eye changes.

Are the .egm files that are skipped by PyFFI only for vanilla Oblivion files?
Is there a different result from right clicking on files/folders to optimize
compared to using the PyFFi Automation Utility?
User avatar
Emma Pennington
 
Posts: 3346
Joined: Tue Oct 17, 2006 8:41 am

Post » Sun Apr 25, 2010 2:12 pm

@Arthmoor: As Pacific Morrowind already asked, can you confirm that .egm related nifs are skipped? (I also checked again and it seems to work as intended...)


Yeah, it skips the opt_geometry spell. Maybe I missed where this was specified, but I was left with the impression that "skip" meant "doesn't touch at all". The files are clearly being touched and having things done to them, just not that spell. At least not on the samples I just ran.

Unrelated: Any word on when the vertex reduction code will be available for the spell that makes _far.nifs?
User avatar
Sweets Sweets
 
Posts: 3339
Joined: Tue Jun 13, 2006 3:26 am

Post » Sun Apr 25, 2010 7:46 pm

@@far327 eyes don't have egms... hair should. PyFFI just looks in the folder to see if there is an egm of %filenamethatitisoptimizing%.egm - if so skip opt_geometry; could you give me a link to XEO5?
Yeah, it skips the opt_geometry spell. Maybe I missed where this was specified, but I was left with the impression that "skip" meant "doesn't touch at all". The files are clearly being touched and having things done to them, just not that spell. At least not on the samples I just ran.

ah yes the opt_geometry is the only spell that has caused problems (at least never been reported otherwise)... so everything else runs and optimizes their segments... but if there are any problems in game let us know and all spells can skip if given another arg.
Unrelated: Any word on when the vertex reduction code will be available for the spell that makes _far.nifs?

Its a real toughie... working slowly on it.
Pacific Morrowind
User avatar
Averielle Garcia
 
Posts: 3491
Joined: Fri Aug 24, 2007 3:41 pm

Post » Mon Apr 26, 2010 2:04 am

Assuming that it's safe to run PyFFi Automation Utlity Alpha 5 on meshes without the need to worry about .egm files anymore, what exceptions should I still add to the pyffi_script.ini if any?
User avatar
Chica Cheve
 
Posts: 3411
Joined: Sun Aug 27, 2006 10:42 pm

Post » Sun Apr 25, 2010 8:06 pm

Assuming that it's safe to run PyFFi Automation Utlity Alpha 5 on meshes without the need to worry about .egm files anymore, what exceptions should I still add to the pyffi_script.ini if any?

I have pyffied all my meshes lately with only the exception "hair". I know it sounds dangerous, but I tell you, it all came out just fine. Do use the new Pyffie 2.1 though. And do have plain, unpyffied versions in reserve in case you do run into a (graphics) issue during play. But a good 100+ hours into my present game, I am quite confident that my pyffie method is valid.
User avatar
sas
 
Posts: 3435
Joined: Thu Aug 03, 2006 8:40 am

Post » Sun Apr 25, 2010 1:52 pm

I wholeheartedly agree with Tommy_H (and extend my thanks to him again :)) for his advice! I've had no problems with using this skip_list:
skip_list	="hair","roothavok"

The first post in this thread says that "roothavoc0#" causes problems if PyFFy-fied, dunno if that got fixed in the latest one though :P (btw, "hair" unfortunately skips "chair" as well :))
User avatar
Jacob Phillips
 
Posts: 3430
Joined: Tue Aug 14, 2007 9:46 am

Post » Mon Apr 26, 2010 3:38 am

Thanks guys, I have PyFFi'd all my meshes once before but avoided anything with a .egm file.
I'll give it another run through over the weekend. ROFL @ "chair"! :biglaugh:
User avatar
Mel E
 
Posts: 3354
Joined: Mon Apr 09, 2007 11:23 pm

Post » Sun Apr 25, 2010 11:23 pm

Afaict, .egms still need to be avoided (the Alpha5 tool makes quick work of that by doing that for you :)) Sorry for not linking to it on the go.
User avatar
Taylah Haines
 
Posts: 3439
Joined: Tue Feb 13, 2007 3:10 am

Post » Sun Apr 25, 2010 9:29 pm

skip_list	="hair","roothavok"

The first post in this thread says that "roothavoc0#" causes problems if PyFFy-fied,

Now I thank you Gratis_monsta, I will take your skiplist then next time! :)

Yeah, I guess the chairs are rather unimportant, I don't think there are many. http://i35.photobucket.com/albums/d191/haegint/Oblivion/Chair_Meshes01.jpg. Just mark them all in the search results folder and pyffie them with the right-click method. :tops:


Edit: I want to stress that the spelling of "roothavok" is indeed the correct one.
User avatar
Tamika Jett
 
Posts: 3301
Joined: Wed Jun 06, 2007 3:44 am

Post » Sun Apr 25, 2010 5:09 pm

Resolved, sorry.
User avatar
Rachel Hall
 
Posts: 3396
Joined: Thu Jun 22, 2006 3:41 pm

Post » Mon Apr 26, 2010 6:49 am

NVM :).
User avatar
Tamara Primo
 
Posts: 3483
Joined: Fri Jul 28, 2006 7:15 am

Post » Mon Apr 26, 2010 12:23 am

Sorry to ask, everyone still seems to be using the alpha tool (which is ok as it is proven to work well!) but the latest pyffi actually *should* make that redundant. Has anyone had success with running pyffi by right click from the Program Files\PyFFI\utilities\toaster\oblivion_optimize.ini file, or is that working for no-one? Particularly it would be nice if someone could confirm that the resume, jobs, and skip options (skip is in oblivion_optimize.ini, the others are in default.ini) are functional.

You may want to make sure that you have a line
skip = hair roothavok
in oblivion_optimize.ini (see comment below).

Here's again the link to the dev version with the .ini file support:

http://www.sendspace.com/file/jtljr6

@PacificMorrowind: Actually, a cheap version of vertex reduction should be very easy to implement: we can just increase the threshold at which vertices are merged in the duplication algorithm. I have a lot on my plate at the moment but I can give it a quick go maybe tomorrow. A more generic algorithm, such as the one in the poly reducer in Blender, would of course be better, but maybe simple vertex merging is enough for far nifs. Testing will tell.

@Tommy_H (and others): I suspect that the problem of the hair nifs is that the triangle ordering must be preserved, which the optimize spell won't do. Do you know what exactly is causing a problem with the optimized roothavok nifs? I'll try to remember to look into these as well.

@Gratis_monsta: You still see corruption / crashes on .egm related nifs, even with the above version?
User avatar
Lucy
 
Posts: 3362
Joined: Sun Sep 10, 2006 4:55 am

Post » Sun Apr 25, 2010 6:12 pm

Amorilia, I'll run a test later with 2.1.3 :) I'll pick the vanilla Armor meshes, as they have some .egm's in there. I'll be back with a report.

@Tommy_H (and others): I suspect that the problem of the hair nifs is that the triangle ordering must be preserved, which the optimize spell won't do. Do you know what exactly is causing a problem with the optimized roothavok nifs? I'll try to remember to look into these as well.

I honestly don't know. I was not aware of the roothavok "issue" before I read it in JdeRau's post. From a technical viewpoint, I still understand really nothing about .nifs and the optimizations and stuff. I'm only fairly decent at making use of PyFFie. Sorry :embarrass:
User avatar
Philip Lyon
 
Posts: 3297
Joined: Tue Aug 14, 2007 6:08 am

Post » Mon Apr 26, 2010 12:12 am

Amorilia, I wanted to go about the test now. But the oblivion_optimize.ini and default.ini are strangely formatted when I open them with the WinXP standard editor. They have no line breaks??!? I manually edited default.ini but I don't dare to touch oblivion_optimize.ini
User avatar
Anthony Diaz
 
Posts: 3474
Joined: Thu Aug 09, 2007 11:24 pm

Post » Mon Apr 26, 2010 5:21 am

Hah, use for instance notepad++ to edit them. They use LF line endings instead of CRLF (I do my development on linux mostly). I'll see if I can fix the installer to have windows style line endings.
User avatar
Portions
 
Posts: 3499
Joined: Thu Jun 14, 2007 1:47 am

Post » Mon Apr 26, 2010 5:01 am

@amorilla: First of all, I'll say thank you for this awesome tool :) It's probably the reason why my FCOM install actually seems to run better than my vanilla installation!!!

I'd never tried messing with the data files before, so I usually try to browse and extract info from the forums before starting. In PyFFY's case, the Alpha5 tool seemed to be the best thing there was when I started to meddle in the Data folder (this was before 2.1.3 btw)

About .egms, this thread contains some posts by Arthmoor and far327 who seemed to have problems with egms and right-click in the latest version. So, I just echoed it :P I haven't tried the 2.1.3 version (other than for a few newly released mod versions ... Reneer's Guard Overhaul for example) And I just tried it on the Unofficial Mod Patch meshes (which I'd already PyFFIed) and the size of the result increased by 86 bytes :) I didn't intend to rePyFFI all my meshes again (because it took an entire week to PyFFI my entire Data folder :P)

On a similar note, if I wanted to PyFFI my meshes in the future, is it recommended to PyFFI the un-PyFFied original meshes or just rePyFFI my already PyFFIed meshes? (so many PyFFIes ... it's starting to get confusing :P)
User avatar
Jinx Sykes
 
Posts: 3501
Joined: Sat Jan 20, 2007 11:12 pm

Post » Mon Apr 26, 2010 2:52 am

...use for instance notepad++ to edit them....

Uhmm, I'm not sure how great the enthusiasm of the general public is going to be if they require an extra editor to set up the .ini files, if on the other hand a comfortable tool like Alpha5 is readily available...

I do not quite understand: Is it at all necessary to specifiy a default path in default.ini? Because further down I seem to be able to specifiy an input directory? From which directory is 2.1.3 going to do the pyffification? I would believe the "in" directory? At least that is what I would like to: Take my bunch of meshes (or single folder, like Armor), put it there and let it run. As is now, I am not sure if pyffie will go to the default path which for me is
C:\Programme\Bethesda Softworks\Oblivion\Data\meshes
and suddenly does all meshes, instead of only the Armor one's from the "in" directory.

I believe that in terms of interface, a bit of polishing would eventually be welcome. But I'm certain that at this point you are more interested in whether the functions work as they should. So I will now download notepad++ just to do the test as I promised I would. But then I think I will revert to Alpha5 again because I feel it is the more straightforward procedure, at this time.

This is how my oblivion_optimize.ini now looks (when viewed in notepad++):

; note: the "oblivion" option, which points to your nifs, is set in default.ini[main]spell = optimizefolder = %(oblivion)s[options]sourcedir = %(oblivion)s; any patterns of files that should be skipped (e.g. \weapons\)skip = hair roothavok

...and the default.ini. Please note I have a dual core. And commented away the entries for FO3 and MW as I don't have them.
[DEFAULT]oblivion = C:\Programme\Bethesda Softworks\Oblivion\Data\meshes;  fallout3  = C:\path\to\your\fallout3\nifs;  morrowind = C:\path\to\your\morrowind\nifs[main]folder = ./in/[options]jobs = 2  ; set this to the number of cores you havesourcedir = ./in/destdir = ./out/resume = Truepause = True


Does that look good? And to actually start Pyffie, I should right-click on the oblivion_optimize.ini, not the Armor test folder? Thanks for a brief comment.
User avatar
Ludivine Poussineau
 
Posts: 3353
Joined: Fri Mar 30, 2007 2:49 pm

Post » Sun Apr 25, 2010 6:09 pm

I just decided to use PyFFI's automater to PyFFI some meshes. It takes a bit of work to set up though.

Resume works properly (had to use it when a PC restart was required).
Setting it to 2 makes PyFFI use both of my CPU cores.
I won't test the skip option. If any meshes give me a problem, I'll just report it here then revert to the original.

Here is my Default.ini file:
[DEFAULT]oblivion = C:\Programs\PyFFI\utilities\toaster\infallout3 = C:\path\to\your\fallout3\nifsmorrowind = C:\path\to\your\morrowind\nifs[main]folder = ./in/[options]jobs = 2; set this to the number of cores you havesourcedir = ./in/destdir = ./out/resume = Truepause = True


My oblivion_optimize.ini file is the same as Tommy's above except the "Skip" section is blank.

Overall, I think this is better than the PyFFI automation tool. And yes, you right-click on oblivion_optimize.ini then click Run with PyFFI after you put the meshes in the In folder.
User avatar
Kayla Bee
 
Posts: 3349
Joined: Fri Aug 24, 2007 5:34 pm

Post » Sun Apr 25, 2010 11:16 pm

My oblivion_optimize.ini file is the same as Tommy's above except the "Skip" section is blank.
Overall, I think this is better than the PyFFI automation tool. And yes, you right-click on oblivion_optimize.ini then click Run with PyFFI after you put the meshes in the In folder.

Thank you JdeRau, you are braver than I am. I'll amend the default path like you did, and will run the test on Armor now. Since you tested without Skip, I'll test with skip, maybe we get different results then.

EDIT: Pyffification running now...
CPU gets a tough workout, getting as hot as it ever got... it svcks up all sytem resources, Task Manager locked at 100% usage :), practically impossible to do anything while this one is running. :)

EDIT2: The lights are going out in my neighborhood... :)

It's not only is toasting the meshes, but the rig as well... Nice though... CPU only gets to like 74 Celsius. Thats only with the bloody Intel boxed cooler (I'm on a E8400 running at stock 3 GHz)
User avatar
hannaH
 
Posts: 3513
Joined: Tue Aug 15, 2006 4:50 am

Post » Sun Apr 25, 2010 8:45 pm

@JdeRau & @Tommy_H: Thanks for the feedback, and confirming that things seem to work as they should. Your ini files look good! Indeed, of course I don't expect people to have to install a new editor just to set up their ini files. I'll definitely sort out this out. I'll also remove the game specific path options, and let it simply default to the ./in folder: I agree that this is more intuitive, and simpler. I'll upload a new installer with fixed ini files tomorrow. I'll also make "skip = hair roothavok" default (just in case, until I've had the time to properly investigate it).

@Gratis_monsta: It shouldn't matter whether you PyFFI the originals or re-PyFFI your nifs. At this moment, there's no point (yet) to re-PyFFI anything (well, almost no point... there have been a few updates but nothing absolutely big). What I'm focussing on right now is improving the user interface and user experience, so effectively we have an "Alpha tool"-like interface - and not just for the optimize spell, but also for all other spells, such as the one to make far nifs, or to modify texture paths, etc. Also, this way all configuration options are configurable in an easy manner, which isn't something you could do with the Alpha tool. Of course I don't know how many people will actually make use of that, but very sofisticated users might appreciate it.
User avatar
Emily Graham
 
Posts: 3447
Joined: Sat Jul 22, 2006 11:34 am

PreviousNext

Return to IV - Oblivion