PyFFI's default skip list doesn't cover things like the havok roots though. At least not yet, one assumes Amorilia is looking to update that since he asked about it.

alright cool. I assume I can do this same procedure with the Nehrim Meshes BSA?

...This thing sure does take quite a while.

Yes, in principle you can run any meshes collection through the toaster.

well, after 5 hours it's finished, but the folders: "distantlod" and "trees" are not in the "out" folder. (even though they were in the "in" folder")

in/distantlod
in/meshes
in/trees

out/meshes

Is that supposed to happen?

Yes it is. The only files PyFFI places in the "out" folder are those that have been modified by said program. PyFFI doesn't touch anything in the '"DistantLOD" or "Trees" folders, therefore, it doesn't write out any modified files for those folders.

great!

One last question, sorry for asking so many.

When I use OBMM to repack the BSA should I choose "Compress All" or "No Compression", or does it not matter?

If you're seeking to squeeze every last drop of performance out, use "no compression" but be sure the resulting file does not exceed 2GB or the game will not be happy about it.

If the BSA only contains the vanilla meshes and trees (ditch the DistantLOD folder if you plan on using TES4LODGen), you'll be fine regarding this.

EDIT: In other news, I gave Arthmoor's optimization method a try today. Took a bit of coaxing to get that massive skip list to work but it worked out pretty well. Thanks for those, Arthmoor (and H2ODK for the regex skip list). :thumbsup:

Sorry if I'm being thick here, but I can't see any "Run Pyffi" in the context menu when right clicking on the ini.
I get the "optimise with pyffi" option when selecting the unpacked mesh folders.

Edit: Tried reinstalling (including python), disabling UAC. Still no option in the context menu.

Edit 2: In essence, how are you meant to use the oblivion_optimize ini?

Alright, the BSA was only 1.2 GBs, so that's good.

I still have more questions unfortunately. Since I'm playing Nehrim exclusively, if I use OBMM's timestamp reset, will that screw up the Nehrim BSA's? (their dates are for 2010, not 2006)

What questions? You need to reset the BSA timestamp after you create you new BSA or the game won't replace loose files with those found in the BSA because the BSA will be newer.

What's the basis for this claim? I'm glad it was brought up, because I was going to ask here but didn't know if it was too off-topic.

Compression drastically reduces File I/O. The BSAs approximately double in size without compression. This means that to load the BSA from the HDD will take twice as long uncompressed as it does compressed. The CPU time to decompress it is probably negligible. OS X (as of Snow Leopard) compresses a great deal of its files for performance reasons. Since most environments these days are I/O limited, even with SSDs, it makes sense to compress what you can. (SSDs compress most content on-the-fly anyway, for higher performance.)

After PyFFI-ing my own BSAs I noticed the hard drive making a great deal more noise than usual, and it seemed to take longer to load. I then switched back to medium compression.

However, I thought maybe when it comes to single-mesh, on-demand access that compression could possibly introduce performance issues, but I find this unlikely. The uncompressed BSA is still one solid file to the OS. This means that it takes up twice the space on the HDD/SSD when uncompressed and takes twice the effort to scan over to rip out an individual file. I of course don't know the technology behind BSAs, but maybe when uncompressed there is a useful "file tree" in the BSA headers that allow the game to jump more quickly to the necessary files? Maybe this is the basis for these claims? I am just positing this, I don't know enough about BSA loading to have any reason for these ideas, just general file I/O knowledge.

Otherwise, I feel like this claim is similar to all the INI tweaks claims that have since been renounced. But I feel like if anybody would know the basis for this claim it'd be you.

The basis is quite simple, though difficult to document with hard numbers. Uncompressed files allow the game to run more smoothly. It's a difference that is noticeable to the point of being more than a placebo effect. Less stutter, even when using OSR. It won't make much difference in terms of FPS though.

It's probably not an Earth-shattering difference which is why it's usually only suggested as a way to squeeze every last drop out. You're trading CPU load during decompression for the file I/O overhead. The file I/O just isn't as much of a burden as the game having to constantly decompress the files - only to find most of the content already compressed anyway since the textures are all .dds format. File I/O concerns also diminish if you keep the drive defragged regularly.

On an SSD I can't really see the difference being anything you could notice as far as I/O since you're then dealing in nanosecond seek times. The CPU burden of decompression might actually play a larger role there. I can't speak to that though as I don't have $900 to spend on an SSD for the sake of a single game.

well, something has gone horribly wrong. After reseting the timestamp on the Oblivion - Meshes.BSA and putting it inside the Nehrim folder, the game crashed. As soon as it finished loading the save it kicked me to the desktop, no CTD or anything. Upon replacing the original BSA the game loads fine.

You're loading in the wrong INI file, I would think. Nehrim can't run on Oblivion's INI file, it needs its own.

Hi, anyone able to shed some light on the "right click on the optimize.ini, and run pyffi" part? I can see no such option.
Using python 2.6.6.
I'm eager to get this in motion so it can go overnight.
:)

Edit: tried with 2.1.7 and 2.1.6 - nothing when right clicking on the oblivion_optimize.ini.
I'm a big noobie at this, and its quite frustrating when it should be a relatively simple process yet I'm not able to follow through with the given instructions.
Oh on W7 64 bit - has that anything to do with it.
UAC is disabled, and Pyffi is not installed in program files.

Help! :brokencomputer:

No, I'm almost positive it's the BSA's. I just tried the PyFF'd Nehrim Meshes BSA, and there were a whole bunch of the yellow exclamation marks.

I must have done something wrong, I just don't know what.

Crap I just remembered, is there some sort of problem with Pyffi if you have the GOTY edition?

Well, you talk about the textures BSA but the topic here was mostly about meshes. The question you addressed was about meshes, and I was really only talking about meshes. DXT5 compression is nothing compared to the compression of archives... I doubt any nanosecond delays from multiple compression will compound, exactly. I also don't see how textures being loaded would be a noticeable cause of stutter. As you're running outside you hardly come across new textures, as the homogeneity of texture assets is pretty high in any given worldspace. If there weren't load screens between worldspaces and exterior/interior I could see texture loading being stuttery. I think a vast majority of the cause is the content streaming code in the engine, including the LOD system, and the grids. It stutters to figure out what to load, not necessarily stuttering from the loading itself. Most of the stuff you come across while speeding across the landscape probably stays in memory anyway (uncompressed I would imagine, but I could be wrong).

I think meshes would be the biggest introduction of stutter if anything, but even then I'm not sure. Meshes compress a lot better, because of the nature of images, but the uncompressed textures are more or less the same average size of the uncompressed NIFs. But then this means the NIFs are loaded from harddrive fairly quickly when compressed and on-the-fly decompression is basically instantaneous at these file sizes. Decompressing a 40KB mesh probably wouldn't register in milliseconds, nor would a 512KB mesh. But a compressed-at-50KB mesh compared to the original 512KB mesh could be loaded from the HDD 10x as fast (not for just one file, but let's say there are 1000 of them, then it approaches 10x as fast).

Anyway, I guess I will just drop my point. I don't believe that anecdotal evidence is evidence in this case. And when it comes to stutter, going simply on observation can't ever be a good measure... The amount you claim it to actually help is within the realm of possible placebo effect simply by your own definition of the benefits (barely any at all). It's not like it's non-OSR vs OSR, as you say. Even still, I get stutter with OSR, but I'm extremely perceptive. I would notice stutter if I were playing from an SSD (I'm sadly not). I'd probably get vertigo without OSR I would imagine. But at least when I played the other day I noticed no difference in stutter with an uncompressed meshes BSA. I guess my point is that I was asking for some hard technological basis, just not the "it's observed to improve performance" argument, That's what all those old INI tweaks used as their basis and now we know that on a technological level they do absolutely nothing.

I would of course accept "observation" if it was scientific in nature. Like recording video (off-screen, FRAPS can't be trusted) and counting the number of discernible stutters between compressed/uncompressed BSAs. But nobody has the time for that of course. I will at least TRY again to decompress the meshes BSA, and additionally decompress the textures BSA, and test it out informally.

Not interested in arguing the point that numerous people have already reported on. You asked for my opinion and I gave it to you based on what I've seen and what others have seen. You won't notice it as much while using OSR, but you WILL notice it if you're as perceptive as you say you are. *I* notice it, and I couldn't tell you if that's just me being overly perceptive or not.

I raised the texture files as an example because it's relevant. Same with nifs. They're not in an easily read format, so there's processor overhead involved in simply decoding them. Adding the burden of decompression on top of all that can be proven to add more time to the process and you're then multiplying that over thousands of files.

And yeah, nobody really has time to give you the kind of proof you seem to be after, so it's either go by what was offered or not. I've personally got better things to do than sit behind my screen with my cell phone and record video stutters and count them and all that.

What was the suggested reasoning for a PyFFid mesh being larger than a non PyFFid mesh? I remember reading it once, but am curious because I am seeing some much larger increases in size that anticipated.

Example - 20mb un-PyFFid and 30mb PyFFid


thanks

Never seen one gain anything approaching that much but the file size increase was just due to some of them getting more raw data but in a more optimized arrangement for the game.

I've just PyFFIed the Cyrodiil NPC Remodeling Projects (all 5 of them) meshes without using the skiplist (so lots of hair and other nasty stuff got pyffied in the out folder ... even egm stuff but I think amorilla said pyffi does what it needs to on them)

Itching to get to Cheydinhal, Bruma and Chorrol to check their blownup faces and infinitely long hairs But from all the Fighter's Guild members I've seen, there's nothing amiss.

Again, apologies for pressing this, but it is doing my head in.
Quoted from tesiv positive and other sources "2. Right-click on the optimize INI and choose "Run PyFFI". Should there be an option in the context menu of ini files?

As far as i know, everything is set up as it should be. I can find no information on this through google or other sources, so I'm assuming I must have done something wrong. What that could be, I have no idea.

I just wanted to ask again, before I attempt to pyffi the vanilla meshes manually.

I had that problem when I changed the default program for the .ini extension to Notepad++ (did you recently install any text editors?)

YES! OH MY GOD! I am using notepad++ :facepalm: :celebration:
I literally spent 8 hours yesterday messing around with this, and scouring all available sources of information on the internet, for solutions.

I cannot express my gratitude enough - Big plate of e-bicuits for you!
[img]http://i12.photobucket.com/albums/a230/Curunen/Misc/23e4ff23.jpg[/img]


Edit: For what it's worth it might be useful to have this info in the OP, because I'm sure I'm not the only one who could hit this stumbling block.