That is the reason, and should also be reflected in the OP. Like Arthmoor said, PyFFI has come a long way and most of the info you would find about it are more likely outdated, unless updated.

Thus also the same reason why previous information regarding PyFFI-cation that didn't reduce mesh size is outdated. I would prefer performance for a minimal file size increase.

EDIT: IIRC, in the previous thread or so amorilia, author of PyFFI, mentioned about these file size increases to be only between 1% and at most 5%. The perspective is that most people nowadays are more gpu-bound (performance) than memory-bound (file size) thus opting for performance. But as mentioned in the log, it would be added in a later version for those wanting minimal file size.

I just PyFFIed the vanilla meshes with 2.1.7 and the two INI files suggested by Arthmoor. I noticed hardly any mesh size increases at all from the vanilla sizes. Certainly most everything over 50KB or so reduced noticeably. The smaller the filesize the smaller the room for optimization, of course.

Since I've seen so many people talking about it taking them 24 and 48 hours, I'm curious if this is the norm? Mine took 1.75 hours to do the meshes folder from the vanilla BSA. I merged the skipped files back in and it's about 100-200MB smaller overall. I generated the PATCH files, but that http://tesnexus.com/downloads/file.php?id=19899 doesn't even work right. I had to trick it into working by doing

XCOPY Original Patches /T /E

On the command line to create an empty directory structure for the patch files to be created in. Otherwise it created a bunch of senseless folder hierarchies using some strange combo of the absolute path and the relative paths ad infinitum. So clearly something in the batch scripts does not work for Windows 7 any longer, I would gather.

So I have a 7zip with the PATCH files but no way for anybody to apply them, as the ApplyBatches.bat script ALSO doesn't work, at least for me.

24-48 hours is not the norm, that's actually pretty extreme. Your results are about average.

Those patch scripts are almost certainly all obsolete as well. There's really no reason to use them anymore because they were built at a time when PyFFI didn't have the intelligent skipping it has now.

Arthmoor, those patch scripts are only used to generate a "diff" between a PyFFI-ed mesh and the original, so that people who can't PyFFI themselves may be able to apply the patch and result in having a PyFFI-ed mesh. Of course only those meshes that differed (i.e. not skipped by PyFFI) will have a corresponding patch, and by uploading these collection of patches one can apply it to their vanilla meshes as well, and end up with a PyFFI-ed mesh equivalent, only that they never did the PyFFI-cation.

This is very helpful for those who doesn't have the time nor experience to correctly PyFFI the mesh. I for one would have wished somebody could have maintiained those patches and updated them to reflect vanilla optimized meshes with the latest PyFFI.

Well, it was still an uncomfortable experience for me. Had to put my fans on high and hope that using 100% of my 8 (logical) CPUs for however long it took wasn't going to explode my computer. I'm on an iMac and the heatsink isn't so great. I was going to cut the jobs in half, but then I didn't know for sure how long it'd take. I had already extrapolated from doing it for 30 minutes that it'd only take 2-4 hours but if it was at the upper range I wouldn't want it taking 8 hours by cutting the jobs in half.

I thought if the patcher still worked I could have at least saved others the trouble. I could remake my own patch script if I wanted, but I don't know if anybody would want it then. Since I guess people have been having to do it manually for so long.

What's ironic is the 7z with the DIFF files is nearly as large as the 7z with the optimized meshes.


Ah, now I'm not sure if he got that or not. I can't really tell either way. But if I could figure out how to remake the batch scripts to actually work, I could upload a new batch patcher with my meshes DIFFs. I'm also looking into whether there are simply better redistributable diff programs that have support built in for recursion. The only reason for the batch files are to work recursively with that "xdelta" program.

I would advise against using any patch files / differential files, like Arthmoor says. Do the Pyffi-fication yourself. One added reason why doing it yourself is you know what you did and do not leave it up to some "blackbox" doing something for you.

Pyffie toasting times are very linear with number of logical cores available and speed of CPU. Quad core is obviously better than single core P4 Northwood from 2003 or whenever.

I want to remind Pyffie users (especially people new to Pyffi) who are on Win7 that there is the potential "leading spaces" issue when unpacking/ copying/ repacking OB vanilla meshes: 13 files in the meshes/dungeons/caves/exterior folder by default have a leading space at the beginning of their file names that needs special attention. Simple copying/ moving them with Win7 Explorer will make the names lose the leading spaces. This is known issue of Win7. It does not affect users on WinXP. I don't know how MacOS is handling this.

For Win7 I recommend to use the .esp workaround plugin that Tomlong has posted on her website. This is probably the easiest way to do it, albeit at the cost of a mod slot (don't know if that .esp is mergeable or not in WB, as I slug it out with Win7 but next time I use the Tomlong .esp).

This has been discussed in the past, but not everybody is reading the whole thread all the time...

http://tesivpositive.animolious.com/?page=pyffi

It's also mentioned in the opening post, but you need to scroll down a bit.

For that matter, PyFFI is nothing but a "blackbox doing something for you". You're not optimizing the meshes by hand or anything. It's also a matter of time. You say how it scales linear with cores, well it also probably scales with the CPU's performance per core. If one i7 core optimizes 50% faster than a p4 core, and you have 4x the cores, it should be 6x faster. If it took me nearly two hours, why should you advise somebody to take 12 hours to do the same thing if a patcher were to be made available? It's like you'd advise against taking the bus to work, after all, you can drive yourself.


All you do is extract the BSA directly to the "in" folder, and create the BSA from the "out" folder. You never move the files and hence it's never a problem. It's even in the http://tesivpositive.animolious.com/?page=pyffi.
...

Anyway, I rewrote the batch files to actually work, I'm not sure how they EVER did work, actually. I don't think it had anything to do with OS differences. And I used a more recent version of the xdelta program too... I've tested it out and it only takes 5 minutes or less to patch the vanilla meshes. If there is any interest I can see about releasing it.

@ jonwd7
You are apparently one of the rare people who follow instructions diligently and carefully. And that's great. But from my experience visiting these boards not everybody is so careful. So I was not trying to lecture you personally but point out a potential stumbling block when unpacking the vanilla - meshes BSA and using Pyffi on it. Some things cannot be stated often enough... just follow the FCOM thread if you need any proof of that.

Regarding this issue of Windows 7 dropping leading whitespace from file names when copying / moving, can the same also be said when using a file copy replacement program, like TeraCopy perhaps? I have no way to confirm this as I have no Windows 7 machine at the moment, but it might help with this particular problem.

Oh yes that is entirely possible. I just don't know which of those 3rd party programs would be capable of keeping the whitespaces under Win7. I'm not using any.

As I see it, the Tomlong's .esp has the advantage that you need not think about doing anything special when you unpack and pyffie the vanilla meshes. Since the vanilla meshes are the only one's where this potentially is a problem it's possible a user may not remember in the future that the vanilla meshes need "special handling". But of course it each user must find his own comfortable way and his own routine how to deal with this particular quirk under Win7.

So does anyone just right click the meshes folder and PYFFI that way? Extract mod, choose meshes folder, right click, pyffi, repack. Any feedback on this procedure?

As for leading spaces, does anyone have the issue with windows 7 when making a copy of the oblivion folder to anothe rlocation, at the end there are always 12 (or 13) files it asks to replace or skip (even though in a copy to a new folder this should not be the case and always the same files)?

It's fine, but I prefer the In/Out method. I wish there wasn't the awful shell integration in the first place. "Optimize with PyFFI" shows on nearly everything... I would rather just have the Run PyFFI option on INI files. I'll just uninstall it after I've PyFFIed everything.

The issue was just discussed a few posts before yours. Read the http://tesivpositive.animolious.com/?page=pyffi. Simply don't move the meshes folders, ever. Unpack, PyFFI, re-pack. If you don't move the files it can't strip the spaces.

That's a good idea! Actually it would be needed only on the oblivion_optimize.ini, right? Maybe Amorilia can do something there for a future version.

So most likely these files are from another mod added loosly to my installation which are meant to overide the Oblivion - Meshes.bsa. Probably UOP but would need to check.

While we're talking about making improvements to the executable, I wouldn't mind an auto-update feature (one that runs when PyFFI is executed, not that is a resident-in-memory type program).
I was disappointed recently to run everything through PyFFI 2.1.5, only to learn a few days later I was way behind!

Ideally, the program would check for the latest version of PyFFI and of Python 2.x, but at least for the latest PyFFI.

http://sourceforge.net/tracker/?group_id=199269 Make a request.

Then there are those of us who not only like it, but pushed to get it made a feature. I'd be highly upset to see it backed out now because I love it. Now, I'd have no issue with there being an option during the install to ask if you want shell integration because that would satisfy everyone. I'd get to keep what I love, you'd get to throw it away without another thought.



This issue is restricted to Windows 7 64bit as far as I know. I don't think anyone even knows why Microsoft did that. Vista 64 users may also have the issue but I've never seen anyone mention it who uses Vista.



Which is why the UOP includes the no leading space versions of the files and specifically directs the resources in the ESP to point to them. Eventually as more and more people end up with Windows 7 on 64 bit machines, the issue would have just gotten worse and worse because it shows up in game as missing meshes if it's not been taken care of.

Arthmoor, what do the characters, and stuff like that, in the parenthesis on your Optimize.ini file do? Like the "?"'s and "i"'s?

Those are all regex codes. I don't recall what the specific things all do but they set the pattern on what to search for and exclude from optimization. I can usually debug a regex with a bit of effort, but trying to write them myself ends up as an exercise in madness.

Need an opinion. Any benefit to running this on QTP3 Redimized? Thanks.

Yes. QTP3-R will see plenty of optimizations.

He, he, heeehhh... so the leading space issue of those 13 vanilla meshes is actually a non-issue (whether Win7-64 or otherwise) if one has the UOP installed, right? Very clever, I like... :thumbsup:

Thanks. Up till now I've just used http://sourceforge.net/projects/pyffi/files/pyffi/ and never clicked on Support. Never noticed a Tracker option before. A bit annoyed at having to set up a Sourceforge account to do so, but oh well.

So, will they help with optimizing my meshes? Will keeping them in the Optimize.ini increase performance?

I'll assume you mean the PyFFI performance, not game performance, and I doubt a regex helps that much if any. Especially with Python, regexes probably have more overhead than simple substr-type tests w/o regex. It's simply a more concise and intelligent way of matching the filenames. "Keeping them" isn't a choice. If you don't have them PyFFI will try to optimize all meshes. This would be bad.

Basically, just copy the two INI files Arthmoor suggested, and use them.

Buf if you really want a glossary:

"." =  Matches anything but a new line (\n)"*" =  Means 0 or more"+" =  Means 1 or more"?" =  Means 0 or 1, or makes the previous "non-greedy" (*?, +?, or ??)"^" =  Means the start of a string"$" =  Means the end of a string"[]" =  Is a set of chars"|" =  Means OR"\" =  Escapes special chars

And there's also craziness like

{m}{m, n}{m, n}?(...)(?...)(?:...)(?=...)  lookahead assertion(?!...)  negative lookahead(?<=...)  lookbehind(?
And then there are about 30 bajillion special characters.  

Basically, I think the point is, "Leave regex alone, it doesn't like to be messed with!"  It's a curmudgeon.