Oh Really?? oo:

Then I humbly suggest that you have been doing something wrong (unless you have a single core CPU) because Pyffi 2.1.6 and 2.1.7 (and 2.1.5 was also IIRC) are definitely capable of using every core at 100%. I just did a re-pyffi last night with my rig in the sig, and Pyffi was still working 100% on each core. :yes:

Hey c0demeist3r
finally I checked the wall behind the smith shop in Anvil. You are correct that one the arcs (the 2nd one from the left of the gate) looks not quite as good as the others. It has a little bit of texture (?) flickering at the top near the high point of the arc and looks to have more of a "seam" in the middle. Certainly the other arcs look "normal good" and this one is a abnormal relative to the others.

However, I reloaded my game and checked with 3 sets of Oblivion - Meshes.bsa:

• vanilla
  
• vanilla 2.1.6 optimized
  
• vanilla 2.1.7 optimized

And the issue was identical in all three instances. So I would be hesitant to blame Pyffi 2.1.6 for this visual glitch. I suspect the wall is not "built" as nicely in the CS in this particular spot. I will check now your screenshot again and may edit this post afterwards.

What textures are you using? I have only all vanilla textures at this time.


Edit: OK jsut looked at your screenie again. I am fairly convinced yours are not the vanilla textures. But I am also fairly convinced we talk about the same wall arc (the 2nd one). So I can "comfort" you saying the issue is present also with the vanilla textures.

I would stand by my above statement that this is an "architectural issue" from the CS. The Beth guy art designer "messed up" when building that particular section of the Anvil wall. Also the other arcs in the wall are all built off the same meshes as the "ugly" arc, another argument why it should not (cannot?) be a Pyffi issue then, IMHO. :no:

Heh, you're right. I've been cheating and using the PyFFI Automator application, and if I just use PyFFI directly it utilizes both cores. Since the Automator's pretty redundant I guess it's time for me to move on

I was about to post my update after having investigated the glitch on Anvil walls myself. You are right that no matter what mesh is used (vanilla, 2.1.6-optimized vanilla, or 2.1.7-optimized vanilla) the glitch is present in all versions, and it would seem PyFFI-ing the mesh didn't reduce the problem (of course I wasn't expecting correction from PyFFI).


Yes it is the same wall and for reference, the affected mesh is the Meshes/Architecture/Castle/Anvil/CastleWall01.NIF.

I am using http://www.tesnexus.com/downloads/file.php?id=18677 on top of http://www.tesnexus.com/downloads/file.php?id=18430 for vanilla. The two really brings out the beauty in the vanilla textures that can even rival QTP. Of course the replacers are for vanilla textures only, so they are not compatible with other texture replacers.


I wasn't really blaming PyFFI, but just merely stating that I noticed the glitch after the process. Though, after investigating further, it is revealed to me that the glitch has been there even before the process began, and I'm happy to have proven wrong before I even began thinking that PyFFI could have messed it up.

I also came across the application, but after having carefully considered the functions of the program and compared it against the install of PyFFI, I have decided to go without the automator instead. Also, for controlling how many cores pyFFI could use you can specify

jobs = {cores}

where {cores} is how many of your CPU logical cores you would like pyFFI to use. By default pyFFI uses the number of CPU logical cores (i.e. the number of physical cores doubled if hyperthreading) in the absence of that setting. You can add it under the [Options] block of the configuration usually found at

{intall-dir}/PyFFI/utilities/toaster/default.ini

You could also control the verbosity of the console output by specifying

verbose = 0 (quiet)verbose = 1 (default: info, warnings and errors)verbose = 2 (noisy)

under the [Options] block of the configuration.

By the way, notice how the folder is appropriately named (toaster)? Well I was thinking not only does it "toasts" meshes, but it could also toast the CPU by utilizing the full logical cores at full load, unless the cooling system is well maintained.

Also, when using the full load of your CPU with pyFFI, you could suspend the process by pressing the Pause/Break key on your keyboard while the console window is active, and resume by pressing any key. In case the process got interrupted by terminating intentionally (ctrl+c to quit) or unexpectedly, pyFFI is configured to resume by skipping already pyFFI-ed meshes and continue the next one.

Noob question: So I often see flickering seams on steps or walkways in cities/outside. So is that a result of screwy meshes or screwy textures? (Sorry, I don't know my ABCs here).

Can Pyffi'ng fix that?

OK, I'm definitely doing something wrong here.
As mentioned above, I'm switching from using the PyFFI Automator application, to just using PyFFI directly. So I have the latest version of Python loaded (64-bit), and PyFFI 2.1.7. Note that this setup does work in the automator.

I extracted all of "Oblivion - Meshes.bsa" into the PyFFI\utilities\toaster\in folder, then right clicked on oblivion_optimize.ini and chose "Run PyFFI".

That's when things went haywire...
I started receiving HUNDREDS of "no handlers could be found..." errors, which from previous threads I know to be harmless. But I thought that message was only supposed to appear a couple times, not hundreds or thousands?
Additionally, python is being launched several times per second, such that hundreds of instances appear in my Task Manager, and all 4GB of my memory and 100% of my processor is being used. And nothing is appearing in the Out folder.

What the heck am I doing wrong? (Or, point me to some documentation on this, as I've had no luck as of yet.)

I think you should only place the "Meshes" folder inside the "in" folder from the "Oblivion - Meshes.bsa" archive. Move the rest to another folder as you will be needing them for recreating the archive.

Also, if you still don't know yet, after the process you need to overwrite the contents of the "Meshes" folder from the "in" folder using the ones from the "out" folder by moving the "Meshes" folder. This is because some of the meshes will be skipped, thus not present in the "out" folder.

I have done my PyFFI-cation using a 32-bit version, so I'm not aware of any subtle difference between a 32- and 64-bit Python. Also, IIRC the PyFFI download from SF doesn't offer a 64-bit version, so you might want to use a 32-bit Python then install pyFFI.

Heh, I bet you're right. PyFFI is probably having problems because it's trying to optimize the LOD and tree files that are also in that BSA!
For now I've switched back to the Automator but will give it another try soon

EDIT: I could also have an issue with the version of Python I'm using. Not only am I using a 64 bit version while PyFFI is 32 bit, but I am using Python 2.7.1 (the newest Python 2.x available), whereas PyFFI's documentation states to use 2.6.6.

About that, I think PyFFI also supports the Python 2.7 series. If you would recall the setup dialog from PyFFI it notifies you of possible components to install, one of which is what version of Python you would be installing it for, from 2.5 to 2.7 IIRC.

Non-meshes folders are not what's throwing PyFFI off, it is programmed to ignore them and it does so quite well. I've tested this several times.

Hmm...perhaps it's a UAC or other security setting problem then? Or, perhaps it's because Python is installed in "Program Files" while PyFFI is in "Program Files (x86)"? I read above that PyFFI should never be installed in Program Files, but then again I've heard the same about Oblivion and yet I've been running it just fine from Program Files. And, as stated before, my setup works just fine if I run PyFFI through the Automator.

One change I will make, is to set jobs = 1 to eliminate the possibility that multithreading is causing issues.

Additionally, I have noticed: the TESPositive documentation states PyFFI's default.ini needs to include the Python program path. But my copy of that file does not have a flag for Python path. I only have "folder", "source-dir", "dest-dir", "resume", and "pause" flags to set.

Any other theories? I'd love to get away from using the Automator but having 500 instances of Python running simultaneously is not particularly helpful Since I have no idea what may or may not be causing the problem, here are my computer's vitals. More info will be provided if requested.

The first thing you migh want to do is revert to a 32-bit Python install. Then install it in a directory not inside the "Program Files". Typically it installs as "c:\python26" for 2.6.6. Then re-install PyFFI on a different directory, say "c:\pyffi". The problem with those guides is that some of the information contained therein are outdated.

You don't have to do any other extra step after installing Python and PyFFI. Of course you can tweak the config files, before you PyFFI your way to Oblivion.

I woudn't suggest sticking with the 64bit Python for one. If you're using Bash, it isn't supported yet, and fiddling with multiple Python installs on the same machine gets dicey.

Second, you don't need to do stuff in the in/out folders anymore either. The OP really needs to get updated with good, solid, actually useful information at some point that reflects where PyFFI has evolved to.

You can extract your meshes into any folder you want, preferably NOT somewhere under Program Files or ProgramFiles(x86) because UAC will beat you to a pulp for that. Then all you have to do is right click on the folder you extracted the meshes to and select "Optimize with PyFFI". One window should open, and it'll sit quietly using up as many cores as you've told that ini file to allow. It'll take a considerable amount of time, but the end result is the same. Also, PyFFI 2.1.7 hasn't displayed any of the previous version's errors or warnings on the vanilla meshes, so it should sail through without any trouble. It even properly skips the .egm and .tri files that come with helms and some hairs. So that's no longer a worry either.

If you have meshes that have already had errors introduced, 2.1.7 can't fix those. You'll need to do vanilla ones again on any it broke before.

If it helps anyone, this is my current Oblivion_optimize.ini file:

[main]; run optimize spellspell = optimize[options]; any patterns of files that should be skipped, being as conservative as possible; (without quotes, separate different regular expressions by a space); at the moment:; - skipping hair nifs (vertex ordering!); - skipping roothavok nifs (not sure why, investigating); - skipping any nif that is known to have an egm or tri associated with it;   find . -name "*.egm" -or -name "*.tri" | sed 'sX.*/XXg' | sed 'sX.tri$XXg' | sed 'sX.egm$XXg' | sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' | sort | uniq | xargs;   (not necessary if all egm files are included, such as in vanilla Oblivion,;   but some mods only include nifs without egm files, and this makes sure;   these cases are handled as well); note: you can safely remove this list if you are sure that you are; optimizing fully extracted folders onlyskip = (?i)(?

That nasty looking skip sequence at the end was something H2ODK came up with and works well, but it needs a slightly altered process to be sure everything gets done. That's where oblivion_optimize2.ini file comes in:
[main]; run optimize spellspell = optimize[options]; any patterns of files that should be skipped; regex expressions; only run on files with these patternsonly = (?i)gnd[.]nif$ (?i)meshes.clutter


When I do an optimization, I use two batch files. #1:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"


#2:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\default.ini" --ini-file="C:\Program Files (x86)\PyFFI\utilities\toaster\oblivion_optimize2.ini" --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"


The first one takes ages to complete, the second usually goes by fairly quick because it's only looking for a specific set of info. What these do though is add a much more robust extra layer of "just in case" so that it REALLY won't touch any helmets and/or hairs and such.

I also have one other used specifically for cleaning any loose _far.nif files:
"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --noninteractive opt_cleanfarnif --dest-dir= --source-dir= --pause --overwrite "%1" > "PyFFILog.txt"


You may also notice that they all drop a log file when they're done. That's handy for quick-scanning for the word error or warning to see if any bugs cropped up. It also reduces the amount of spam from the handler messages.												

@Arthmoor: Gotta hand it over to the pro. Thanks for the tips. Nice regex.

Anithinks, what videao card do you have and what drivers version? If AMD / ATI, then it is quite likely that the flickering is coming from the video card - or the CCC drivers rather.

It could be far simpler than that even. Flickering can happen when a mesh is slightly overlapping another mesh. The sort of thing the UOP generally fixes.

The same effect can happen if you get two of the same mesh placed on top of each other.

Ah! OK... then please if you're into it sometimes, look at that city wall arch in Anvil I've been discussing with c0demeist3r (just a few posts above this one). It's the arch behind Moravyn's (sp?) smith shop, the 2nd one to the left of the main city gate. There is flickering there towards the top of the arch, and there is kind of a "seam" where the halves of the stonewall come together. The other arches all look "normal" and do not flicker. It's admittedly a very small issue, maybe not worth the effort to rectify anything.

Maybe if there will be another UOP update this one might find it's way into it? Thanks so much! :thumbsup:

As to flickering from ATI video drivers: I was just mentioning it in case he got flickering in a spot where there is no meshes issue. That type of "ATI flickering" was present often enough on my former ATI 3870 card. Don't know about later models, although I believe the issue is still present today to some degree at least if one can believe the PC magazines, depending also on the game in question. OB was (is?) one affected by it, unfortunately.

Well, you were definitely right about using 32-bit versus 64-bit. I redid my setup for 32-bit Python, and ran the already optimized meshes through again. At least a quarter of the meshes were further optimized when I did so, leading me to believe some of the optimization "spells" PyFFI was trying to use weren't available to it in the 64-bit version.
That said, I still haven't been able to figure out why I had hundreds of Python processes launching at once. I'll have to keep playing with it...


Oh, and Arthmoor--thanks for the tips! A couple questions:
(1) You appear to run PyFFI on your _gnd files, including helmet_gnd.nif. I assume it's safe to do so then?
(2) I see that you also removed "emperor" from the default exclude list. Any idea why it was on the list to begin with?
And a suggestion: This may already be accounted for in your setup as I don't know how to read your default exclude list. But you may also want to include the strings "chair", "monkshood", and "brotherhood" in your second batch, since I think they would be excluded from the first batch file! I get around the problem by renaming these before running PyFFI.


EDIT: OK, I have another question now. Running the vanilla meshes through 2.1.7 is actually increasing most mesh sizes, which I did not expect. 2.1.5 decreased them by 15% or more. Is this to be expected? If so, is this due to the comment in the 2.1.6 changelog that triangulation (which improves FPS but also increases file size) rather than stripification (which reduces file size but can decrease FPS) is always used? This may be a topic to add to the FAQ.

The regex for the exclusions isn't my doing. I only have a very basic grasp on the wizardry involved in such things. H2ODK is the one who came up with those and they work rather well.

It's safe to run PyFFI on the _gnd meshes because they don't have to contend with the geomorph data the normal helmets do. That's why the second batch file goes back and only optimizes those, because they'll otherwise get skipped by the first one.

As far as I've been able to tell, "monkshood", "brotherhood", and "chair" are all processed. If not, it's probably not going to have much of an impact.

Increased file size is not an issue. It's not important how large the files are, it's important how well the geometry within them gets optimized. 2.1.7 does a much better job with that and with the collision geometry. Yes, it's using the triangulation method that performed best during the 2.1.6 testing.

I don't know why emperor was on the list before so I can't help you there.

Arthmoor's customized Oblivion_optimize.ini post really needs to be quoted in the first post and the first post of every new PyFFI thread. That info was incredibly useful, thank you Arthmoor.

Tommy_H: I have an NVidia GTX 480 with the latest drivers, so not an AMD/ATI issue. Maybe its because of what Arthmoor listed, the overlapping meshes.

I'll try Pyffi'ing the meshes with v 2.17. If I still see the issue, I'll report it here/UOP thread. But frankyly, it is a minor annoyance in the grand scheme of things - I'm much too grateful to run my 390 mods together (254 when merged/deactivated etc.)

When repacking the Oblivion meshes bsa how do I get the trees folder (the trees folder with all the .spt files) to go in using the OBMM bsa creator? For some reason I can't get it to add them in.

Edit: Nevermind, got it to work finally.

Wish granted.

Arthmoor, I could be mistaken, but I think removing these terms from your first configuration file:

(?i)gnd[.]nif$ (?i)meshes.clutter

Would eliminate the need for the second run entirely. It looks like those codes, which are in the skip list for #1, are exactly the same as those in the include list for #2. And the commands that you are executing with both config files are exactly the same.

EDIT: Nevermind, I see now why the two runs are necessary. It is so that NIFs such as helmet_gnd.nif, which include terms both on the "skip" list and on the "include" list, are PyFFId.