[info][relz] PyFFI - Python File Format Interface

Post » Fri May 20, 2011 1:42 pm

pyffi.toaster:WARNING:Found an insane amount of 39474 triangles in geometry: consider simplifying the mesh or breaking it up in smaller parts.

https://sourceforge.net/tracker/?func=detail&aid=3058121&group_id=199269&atid=968816

Good thing in PyFFI Automation Utility that it always checks for size of optimized nif. If its bigger than vanilla (my multiplier is set to 1 (default is 0.975) then it skips it. But I guess fail optimization is reported by PyFFi itself. So I have alot of skipped nifs, but only 3 erroneous nifs so far. Therefore Automation Utility is a must have if you dont want to let Pyffi make worser result than vanilla nif. And of course support for multicores and resume support after crash or closing the utility makes it must have even more. But most important feature is a comparsion of optimized and vanilla nifs of course.

I don't think you can judge if a nif perform better by the size of it.

You also don't need the Automation Utility for multicore support, or resume (if run with in/out folders it will by default skip files already in the out folder)
User avatar
SiLa
 
Posts: 3447
Joined: Tue Jun 13, 2006 7:52 am

Post » Fri May 20, 2011 2:51 am

Actually the automation utility has been outdated and obsolete for some time. At least since PyFFI got the ability to right-click a folder and optimize directly from it. A smaller file size means nothing. It's more important for the internal structure to be streamlined, and sometimes that will result in slightly larger file sizes.
User avatar
Romy Welsch
 
Posts: 3329
Joined: Wed Apr 25, 2007 10:36 pm

Post » Fri May 20, 2011 3:40 am

Ah, ic. But what about Pyffi failures? Theyre logged somewhere? So when I'll resume optimization I can add them to skip list.
User avatar
luis ortiz
 
Posts: 3355
Joined: Sun Oct 07, 2007 8:21 pm

Post » Fri May 20, 2011 11:26 am

It works, does not cause a crash :)

Thank you. I guess I'll go through all the meshes and head for a 2.61 release.
User avatar
Hella Beast
 
Posts: 3434
Joined: Mon Jul 16, 2007 2:50 am

Post » Fri May 20, 2011 1:12 pm

Hmm, marker_error.nif, chorrolhousemiddle06interior.nif and chorrolhousemiddle07interior.nif are optimized (at least partially) and copied to \out folder while Automation Utility doesnt allow them to pass further. But wheres the full log file of processed files? I feel like blind without it.
User avatar
Matthew Barrows
 
Posts: 3388
Joined: Thu Jun 28, 2007 11:24 pm

Post » Fri May 20, 2011 4:14 am

The only way to make a log is to redirect the output to a txt file.
User avatar
Mashystar
 
Posts: 3460
Joined: Mon Jul 16, 2007 6:35 am

Post » Fri May 20, 2011 4:54 am

python.exe Scripts\niftoaster.py optimize D:\Program Files\PyFFI\utilities\toaster\oblivion_optimize.ini > C:\log.txt

This is correct command, so I will have resume support and processed files will be saved in \out?
User avatar
keri seymour
 
Posts: 3361
Joined: Thu Oct 19, 2006 4:09 am

Post » Fri May 20, 2011 10:28 am

python.exe Scripts\niftoaster.py optimize D:\Program Files\PyFFI\utilities\toaster\oblivion_optimize.ini > C:\log.txt

This is correct command, so I will have resume support and processed files will be copied to \out?


change it to
C:\Python26\python.exe C:\Python26\Scripts\niftoaster.py --ini-file=default.ini --ini-file=oblivion_optimize.ini > log.txt

Change the Python install dirs to where you have yours installed.
Run in PyFFI\utilities\toaster folder (where the ini's are)
User avatar
Beulah Bell
 
Posts: 3372
Joined: Thu Nov 23, 2006 7:08 pm

Post » Fri May 20, 2011 7:59 am

Sorry for another lame question, but how to run it in PyFFI\utilities\toaster folder?
UPD: ok i made a .bat file there but damn command line has a problem with understanding full windows names. whats the DOS name for Program Files folder?
UPD: i just used quotes for it. Works now. Thanks!
User avatar
Siobhan Wallis-McRobert
 
Posts: 3449
Joined: Fri Dec 08, 2006 4:09 pm

Post » Fri May 20, 2011 9:23 am

So, I optimized all nifs with PyFFI 2.1.5 and tested perfomance on my old PC, my modern pc burned a while ago so I use this one currently: Pentium 4 1.5 Ghz, 1.25 Gb of ram, Radeon 9600 XT 128 mb ram.

vanilla (fort nearby imperial city):
running game: 7-9 fps
console open: 14-15 fps

optimized and packed (uncompressed) in new bsa:
running game: 6-7 fps
console open: 11-12 fps
--
vanilla (tiber septim hotel):
running game: 12-13
console open: 13-14

optimized and packed (uncompressed) in new bsa:
running game: 12-13
console open: 13-14

Looking at results looks like that perfomance is degraded in exterior. And no any speedup in interior. So, its not useful to run PyFFI on vanilla meshes for those who want to have higher framerates.
User avatar
SexyPimpAss
 
Posts: 3416
Joined: Wed Nov 15, 2006 9:24 am

Post » Fri May 20, 2011 6:16 pm

It's not useful in your case to PyFFI the meshes, but many of us have seen a substantial boost in performance, and there's a few reports from the 2.1.6 testing that suggest it will improve again once the vertex caching version is released. Besides, looking at the stats for the system you're testing this on borders on being invalid since it BARELY qualifies for the minimum system specs.

Also, the Tiber Septim Hotel is a bad location to do testing. You should be using someplace with more consistency of content, not prone to NPC disruptions. Somewhere like the Elder Council chambers, or a shop that's only staffed by a couple of NPCs.
User avatar
Donald Richards
 
Posts: 3378
Joined: Sat Jun 30, 2007 3:59 am

Post » Fri May 20, 2011 1:20 pm

It's not useful in your case to PyFFI the meshes

Why not. If it optimizes meshes then no any degraded perfomance must be on any video card which is supported by Oblivion. But so far tests shows otherwise.

but many of us have seen a substantial boost in performance

Its been carefully tested as well or just was checked with eyes only? This could be a placebo effect. I'm talking about only vanilla meshes optimization. I think PyFFI helps alot in some cases (like with buggy nif exporter in 3ds max).

Besides, looking at the stats for the system you're testing this on borders on being invalid since it BARELY qualifies for the minimum system specs.

Only cpu. Which is slower on 500 mhz than minimal requirements - 2 ghz. Even better for testing Pyffi, because it becomes more noticeable.
Here is official minimum system reqs: http://www.elderscrolls.com/games/oblivion_faq.htm
Also I have mid to high video settings.

Also, the Tiber Septim Hotel is a bad location to do testing. You should be using someplace with more consistency of content, not prone to NPC disruptions. Somewhere like the Elder Council chambers, or a shop that's only staffed by a couple of NPCs.

Not in this case. I tested it in apartments and there were no any npcs. Also thats why theres a result with enabled console, because it stops any script/ai processing.

You should be using someplace with more consistency of content, not prone to NPC disruptions.

That was the first test. Outside and alot of polys. No NPCs in loaded cells as well.
User avatar
Sasha Brown
 
Posts: 3426
Joined: Sat Jan 20, 2007 4:46 pm

Post » Fri May 20, 2011 2:53 pm

It's been tested by plenty of people, Kivan included, over a very long period of time. You come along and post contradictory information on a system barely able to run the game to being with and expect everyone else to simply drop what they know to be true based on that? You're making a pretty bold claim about something that's been proven to work with great effect for a huge number of people, burden of proof isn't on us.

Also, with 128mb if VRAM, you're in a situation known to exert disk access, which will skew the results in bad ways. Running that with all sliders maxed isn't really helping you.
User avatar
LuBiE LoU
 
Posts: 3391
Joined: Sun Jun 18, 2006 4:43 pm

Post » Fri May 20, 2011 6:13 pm

It's been tested by plenty of people, Kivan included, over a very long period of time.

Can you link their results? Not words, but numbers.

You come along and post contradictory information on a system barely able to run the game to being with and expect everyone else to simply drop what they know to be true based on that? You're making a pretty bold claim about something that's been proven to work with great effect for a huge number of people, burden of proof isn't on us.

Just saying what I experience with current config with vanilla meshes optimization. How this can be called optimization if perfomance is degraded? No matter what system is it tested on. I dont say that performance is degraded for everyone, but if it works like that for this config I guess I'm not only one.

Also, with 128mb if VRAM, you're in a situation known to exert disk access, which will skew the results in bad ways. Running that with all sliders maxed isn't really helping you.

I have extended AGP memory in bios on 128 mb, so its 256 overall. And results are captured very carefully, without running around, same viewing direction and after everything is loaded in first few seconds. I have only trees and npcs sliders maxed.

I will test it on new hardware as well soon. Getting Core i5 and mid-level Radeon from HD 5xxx series, but I suppose I will not notice any difference on this system.

Heres my oblivion.ini file. http://rghost.ru/2926785/private/c3c34ada30545db207b2b1a1f960757a You can load it up in your game and check. Many graphical features are disabled.
User avatar
Annick Charron
 
Posts: 3367
Joined: Fri Dec 29, 2006 3:03 pm

Post » Fri May 20, 2011 10:19 am

I have extended AGP memory in bios on 128 mb, so its 256 overall. And results are captured very carefully, without running around, same viewing direction and after everything is loaded in first few seconds. I have only trees and npcs sliders maxed.

Sharing system RAM is not the same as having 256MB VRAM.
User avatar
aisha jamil
 
Posts: 3436
Joined: Sun Jul 02, 2006 11:54 am

Post » Fri May 20, 2011 3:10 am

Sharing system RAM is not the same as having 256MB VRAM.

I know. But even 128 mb of main vram is only half used on load. Its easy to check with ATI Tray Tools. First test in exterior - free video (vram) memory: 63 mb, free texture memory: 181 mb, considering also that I have right now Firefox 3 running with few sites opened.
User avatar
x a million...
 
Posts: 3464
Joined: Tue Jun 13, 2006 2:59 pm

Post » Fri May 20, 2011 5:06 pm

Can you link their results? Not words, but numbers.


Do you have any idea how long its been since they tested this? I can't speak for them directly at this point. The threads they posted those results in are long gone.

I can speak for my own results though, and they contradict yours. I posted anywhere from a 5-10 FPS gain upon first using PyFFI on the vanilla meshes.


How this can be called optimization if perfomance is degraded? No matter what system is it tested on. I dont say that performance is degraded for everyone, but if it works like that for this config I guess I'm not only one.


You are using a configuration that was horribly out of date even 5 years ago, and as indicated is based on AGP, with a shared memory pool. How can you honestly expect those results to be of any value to people who are using more recent hardware?

I will test it on new hardware as well soon. Getting Core i5 and mid-level Radeon from HD 5xxx series, but I suppose I will not notice any difference on this system.


You will likely find yourself surprised at the difference. I know I was, because my system specs blow this game's listed requirements clean out of the sky.

Many graphical features are disabled.


Ironically, this may be why you're seeing degraded performance. Oblivion is an oddball, disabling some things can actually harm performance rather than help.
User avatar
Peter P Canning
 
Posts: 3531
Joined: Tue May 22, 2007 2:44 am

Post » Fri May 20, 2011 2:04 pm

I can speak for my own results though, and they contradict yours. I posted anywhere from a 5-10 FPS gain upon first using PyFFI on the vanilla meshes.

In which locations? Your system info? Maybe not every nif is optimized as well as can be and some of them are degrading? I'm ready to help to improve optimization algorithm not only for new systems, but also for old - in this way it will be effective for everyone. You actually are damn sure that you have gain everywhere and dont have lower than vanilla perfomance in some places?

Ironically, this may be why you're seeing degraded performance. Oblivion is an oddball, disabling some things can actually harm performance rather than help.

Tests were ran with exactly same config.

You are using a configuration that was horribly out of date even 5 years ago, and as indicated is based on AGP, with a shared memory pool. How can you honestly expect those results to be of any value to people who are using more recent hardware?

AGP is not bad as you think. Many current-gen mid-range cards were released on AGP - Radeon HD 4xxx series for example. AGP memory is not used at all if a free vram remains, so while everything is stays in vram, agp memory pool isnt active. I can even disable it completely and test again if you want. 5 years ago it was a mid-range config, cept for processor. Check system requirements for these games (random pick of highly rated games back then, which I've played): TrackMania Sunrise, SpellForce: The Breath of Winter, Need for Speed: Most Wanted, The Movies, Splinter Cell: Chaos Theory, SWAT 4, F.E.A.R., which has been released in 2005. SpellForce: The Breath of Winter for example.

Hardware Requirements
Intel Pentium ? III 1GHz
256 MB RAM
3D-Graphic card with 32 MB RAM (compatible with GeForce2 or better)
Windows? 98, ME, 2000, XP
DirectX?9.0a or higher
2 GB free fixed disk storage

Recommended

Intel Pentium? 4 1,8 GHz
512 MB RAM
3D-Graphic card with 64 MB RAM (compatible with GeForce4 Ti or better)
2 GB free fixed disk storage
Windows? 98, ME, 2000, XP
DirectX? 9.0a or higher
User avatar
Shianne Donato
 
Posts: 3422
Joined: Sat Aug 11, 2007 5:55 am

Post » Fri May 20, 2011 2:21 pm

Unfortunately https://sourceforge.net/tracker/index.php?func=detail&aid=2994654&group_id=199269&atid=968813 show that PyFFIing vanilla meshes ATM, does not improve performance on older systems.
User avatar
Haley Merkley
 
Posts: 3356
Joined: Sat Jan 13, 2007 12:53 pm

Post » Fri May 20, 2011 1:20 pm

Those tests were conducted on the upcoming 2.1.6, correct? That version isn't available for general download yet as far as I can tell. So Rancen is using the one without the vertex caching algorithms. That's the one pretty much everyone I've seen using it gets solid results from.

@Rancen:

The system requirements listed on the box copy of Oblivion are:
* Windows XP
* 512 MB RAM
* 2GHz CPU
* 128MB video card, capable of DX9.0 or better.

You're already well below the CPU threshold, and sitting right on the video threshold. Yet you're making blanket claims that PyFFI is useless for the vanilla meshes based on that. This has not been proven to be the case for the vast majority of users.

The testing H2Odk is referring to is for a version that hasn't yet been officially released.
User avatar
April
 
Posts: 3479
Joined: Tue Jun 20, 2006 1:33 am

Post » Fri May 20, 2011 2:37 pm

Unfortunately https://sourceforge.net/tracker/index.php?func=detail&aid=2994654&group_id=199269&atid=968813 show that PyFFIing vanilla meshes ATM, does not improve performance on older systems.

I don't understand this - doesn't the user (UndWDoG or antonio16) that reported this issue (H2ODK's link) have a really high-end system? There was a performance decrease for him as well...

And the status for that issue is "fixed" and it is closed now. So is this resolved or what?
User avatar
Elisabete Gaspar
 
Posts: 3558
Joined: Thu Aug 31, 2006 1:15 pm

Post » Fri May 20, 2011 1:31 pm

I assume itll take a while to optimize a data folder? 24.2Ggigs in the data folder atm with 53,420 files and 4,445 folders. Its already been 15-20 mins.
User avatar
Elisha KIng
 
Posts: 3285
Joined: Sat Aug 18, 2007 12:18 am

Post » Fri May 20, 2011 1:39 pm

Those tests were conducted on the upcoming 2.1.6, correct? That version isn't available for general download yet as far as I can tell. So Rancen is using the one without the vertex caching algorithms. That's the one pretty much everyone I've seen using it gets solid results from.

Correct. I am optimizing the vanilla meshes with 2.1.5 ATM so I can compare that to the unoptimized (with the same system and saves as those tests).

EDIT:
I did the same tests (with the same saves) as I did https://sourceforge.net/tracker/index.php?func=detail&aid=2994654&group_id=199269&atid=968813

SETUP 2
Pentium P4-640 Prescott 2M @3.2GHz
2 GB ram
ATi Radeon HD 4350 512MB @1280x1024

save 1 (12 NPC in scene)
15.800, 15.767, 15.683 (-7.722 fps)
save 2 (0 NPC in scene)
26.433, 26.567, 26.517 (-3.422 fps)
User avatar
Robert Devlin
 
Posts: 3521
Joined: Mon Jul 23, 2007 2:19 pm

Post » Fri May 20, 2011 10:23 am

save 1 (12 NPC in scene)
15.800, 15.767, 15.683 (-7.722 fps)
save 2 (0 NPC in scene)
26.433, 26.567, 26.517 (-3.422 fps)

Another proof. So this issue of fps drop is vertices optimization in 2.1.5. And new vertex cache triangles method in 2.1.6 in case of old systems (overall as well) works best and gives fps increase. Thanks for testing on many different systems. I will test it soon.

And the status for that issue is "fixed" and it is closed now. So is this resolved or what?

Looks like it. With new vertex cache triangles optimization.

You're already well below the CPU threshold, and sitting right on the video threshold. Yet you're making blanket claims that PyFFI is useless for the vanilla meshes based on that. This has not been proven to be the case for the vast majority of users.

Proven? Have you tested each system of vast majority of users personally? Or they all showed test results? Many (or even most) of them I think even havent tested it by just believing that it works better for some. And placebo made the rest.
User avatar
BrEezy Baby
 
Posts: 3478
Joined: Sun Mar 11, 2007 4:22 am

Post » Fri May 20, 2011 7:19 am

Another proof.


Proof that having more NPCs on screen lowers frame rates, yes. I doubt anyone is going to dispute that. Proof of PyFFI causing it? Not unless the result you're quoting had 12 NPCs on screen in each case and performed worse for it.

That said, Amorilia has already confirmed the vertex caching method is better all around, and especially at handling NPCs, which appears to be one area the current methods don't handle as well as they could.

Proven? Have you tested each system of vast majority of users personally?


Unless you're prepared to call everyone who says it performs as advertised a liar, I'd back off from that line of reasoning.
User avatar
TIhIsmc L Griot
 
Posts: 3405
Joined: Fri Aug 03, 2007 6:59 pm

PreviousNext

Return to IV - Oblivion