[WIPz] Fallout Stutter Remover (FSR)

Post » Mon Feb 01, 2010 1:09 pm

I just want to say thanks a lot to SkyRanger-1! You've made this game playble for me. Personally, I think Bethesda should give you a stipend for this mod. Before installing the beta version, I had the 4htz beat frequency issue which gave the game the dreaded micro stutter. Now its very smooth. One note I have for you regarding the mod:

I changed the max frames/sec from 30 to 60. Interestingly fraps says that my frame rate hovers around 63-62 FPS occasionally dropping to 58 with a bit of lag. NOTE: My desktop refresh is 75 hts, but when I set the max FPS to 75 it hovers around 78-76. I'll post the logs shortly.

Thanks again for your work!
User avatar
Kelly Upshall
 
Posts: 3475
Joined: Sat Oct 28, 2006 6:26 pm

Post » Mon Feb 01, 2010 12:15 am

Anyone getting a lot of freezes from Fallout? I'm wondering just how common deadlocks are with FO3. I just ran in to what looks like one and may need to kill it somehow to proceed with my light-critical-section tweaks, as I'm triggering it almost instantly atm.
User avatar
brandon frier
 
Posts: 3422
Joined: Wed Oct 17, 2007 8:47 pm

Post » Mon Feb 01, 2010 6:06 am

Anyone getting a lot of freezes from Fallout? I'm wondering just how common deadlocks are with FO3. I just ran in to what looks like one and may need to kill it somehow to proceed with my light-critical-section tweaks, as I'm triggering it almost instantly atm.


Most freezes for me have been random, and not very often.
BUT recently my game developed a reproducible CTD when approaching Jefferson Memorial.
And I tend to get rather many crashes when aproaching Rockbreakers last Gas, which I suspect to be somewhat connected to MMM.

Anyway I read somewhere regarding the Jefferson crash that the gamedata for that location sometimes seems to get corrupted and there isn't a whole lot you can do about it once it's happened.
Can you tie your crash to a special location? Did you try leaving the area?
User avatar
Charlie Sarson
 
Posts: 3445
Joined: Thu May 17, 2007 12:38 pm

Post » Mon Feb 01, 2010 5:07 am

Personally, I find that F3 crashes rather often... though usually randomly. I don't know why, though. Once, I got past one by equipping a different set of armour... even though changing around my mods didn't help! :blink:

Small question - what does the critical section suppression tweak do? While there were explanations in OSR, they seem to have nothing to do with the ones here, in terms of effect. (I've been using '63' for it, and it works rather well... and doesn't seem to be too unstable, oddly enough.)
User avatar
Liv Staff
 
Posts: 3473
Joined: Wed Oct 25, 2006 10:51 pm

Post » Mon Feb 01, 2010 1:29 pm

I've solved my deadlock problems. They were my fault, not Fallouts. Not to say that Fallout doesn't have similar issues of its own.

Personally, I find that F3 crashes rather often... though usually randomly. I don't know why, though. Once, I got past one by equipping a different set of armour... even though changing around my mods didn't help! :blink:

Small question - what does the critical section suppression tweak do? While there were explanations in OSR, they seem to have nothing to do with the ones here, in terms of effect. (I've been using '63' for it, and it works rather well... and doesn't seem to be too unstable, oddly enough.)

Critical section suppresion suppresses particular bits of thread-safety stuff in Fallout. The value you give it is a bitfield, meaning that 63 is interpreted as the combination of 1,2,4,8,16, and 32. 1 tells it to suppress the CS at Renderer+0x180 (which I recommend), 2 tells it to suppress the CS at Renderer+0x80 (which I don't recommend - the performance difference is minor and some people experienced instability with that setting on Oblivion). 4, 8, 16, and 32 have no effect on Fallout, but did have effects (not recommended) on Oblivion. Or that's what it means in that version. A new version is coming out very shortly which uses a rather different format for specifying these things.
User avatar
Lalla Vu
 
Posts: 3411
Joined: Wed Jul 19, 2006 9:40 am

Post » Mon Feb 01, 2010 5:41 am

For a while OSR/FSR versions labeled "beta" have been functioning as release versions, while versions labeled "alpha" have been functioning as betas. Now is the time for this to bite me on the ass.

There is a new extra-alpha-ish version up at:
ftp://71.115.222.171/sr_Fallout_Stutter_Remover.dll

The old alpha is temporarily still available:
ftp://71.115.222.171/old_alpha/sr_Fallout_Stutter_Remover.dll

Most of the big things are the same, but...
1. The configuration file is now COMPLETELY different. Do not try to use an old ini file with the new version.
2. The configuration file now has significantly more ability to adjust the behavior of FSR. This includes some effects that in theory might allow performance improvements, like the ability to specify separate modes for specific critical sections. It also includes some additions intended to make it friendlier - at the top of the file is now a section that lets you do things like turn on or off console output from the dll, or disable or enable the each set of hooks this does (ie FPS management, CS hooks, LCS hooks, heap hooks (which are still broken), hashtable hooks, fastexit, 64 hertz fix, and (not actually a hook) scheduling resolution).
3. The Light-Critical-Section hook is the only one with major raw capabilities changes: LCS exiting is now hooked in addition to LCS entrance, which allows a lot of the limits of the old LCS code to be bypassed. I had to get a bit brutal to hook LCS exiting, but it seems to be working. The LCS code can now detect contention properly, remember per-LCS-object spincounts and modes, etc.
4. Profiling is working better than in the last alpha IIRC.
5. Hashtable code was tweaked I think.
User avatar
Frank Firefly
 
Posts: 3429
Joined: Sun Aug 19, 2007 9:34 am

Post » Mon Feb 01, 2010 2:30 pm

I'm not able to download anything that you upload on this ftp site... .

Can you please post these links on an http site. Thanks.
User avatar
Cheville Thompson
 
Posts: 3404
Joined: Sun Mar 25, 2007 2:33 pm

Post » Mon Feb 01, 2010 2:26 pm

I'm not able to download anything that you upload on this ftp site... .

Can you please post these links on an http site. Thanks.

It's now, at least temporarily, available on the fallout3nexus page, under optional downloads as "alpha".
http://www.fallout3nexus.com/downloads/file.php?id=8886
User avatar
Horse gal smithe
 
Posts: 3302
Joined: Wed Jul 05, 2006 9:23 pm

Post » Mon Feb 01, 2010 4:59 am

Some of the technical stuff in that release that will probably just confuse you:

If you set CriticalSections\bEnableProfiling to 1 and LightCriticalSections\bEnableProfiling to 1 in the ini file then the game will periodically print summaries of critical section performance to the log file. A summary might look like this:
Critical Sections summary(PD3): time  409   tt:  2.4s   tc:17k    dynamic LCS 035B0044      1241ms   1082     1147us   20ms -99   86D7EA   86D94C      static CS  011792B4       218ms    244      893us    5ms   1   BD3B65   BD1DA2   BD3F32   BD3D78 ...     static CS  01179690       110ms    440      251us  110ms   1   BECEA1   BEE098   BF35DD     dynamic CS  1808444C       161ms    164      981us   27ms   1   8284F7   878F95   876E9D   657FE5 ...    dynamic LCS 035B10AC       116ms     58     1988us   20ms -99   86D7EA   86D94C     dynamic LCS 035B2114        98ms     58     1692us   20ms -99   86D94C   86D7EA      static LCS 0116FCA0        68ms    126      536us   23ms -99   AE428D   AE6710     dynamic CS  1808454C        67ms     29     2290us   10ms   1   828509   873172   88B93C     dynamic LCS 035B3A4C        39ms      7     5495us   20ms -99   86D7EA   86D94C     dynamic LCS 035BCA5C        38ms     18     2113us   19ms -99   86D7EA   86D94C     dynamic LCS 035B297C        38ms     13     2901us   19ms -99   86D7EA   86D94C      static LCS 0106E5A0        34ms     12k       3us   10ms -99   42364D   58791B   AE7D33   ADE578 ...     static LCS 0108EDE0        21ms     29      704us   19ms -99   8268B8   82735D   84B6D1     dynamic LCS 035B7924        19ms      3     6061us   19ms -99   86D7EA   86D94C     dynamic LCS 035B531C        19ms     14     1329us   19ms -99   86D94C   86D7EA     dynamic CS  035A6250        27ms   1459       18us    6ms   4   86E55A   86E23A      static CS  01073AB4        19ms     43      444us   10ms   1   58A85B   591488   58D262   58D104 ...    dynamic CS  18FA3CDC        25ms    286       89us    3ms   1   BCF421   BCF1B8     dynamic CS  181467CC        17ms      4     4056us   11ms   0   BCB86B   BC9DBE     dynamic LCS 035C8CDC         3ms     43       64us    3ms -99   86C9D5   86C877 

The form is:
A: "static" or "dynamic"
B: "CS" or "LCS"
C: object address in hexadecimal
D: total time spent blocking on it, in milliseconds
E: number of times it was blocked on
F: average time spent blocking on it, in microseconds (ie the previous two divided by each other)
G: the single longest time spent blocking on it, in milliseconds
H: the highest number of threads blocking on it at once (currently misreported as -99 by LCSes)
I: a list of caller addresses in hexadecimal, in the order that they were detected; if more than a few are detected it will print the first few then a "..." to indicate that too many to print were found

Generally, the total time a specific CS or LCS spent blocking ("D" in the list above) is the biggest indicator that it may have been responsible for performance issues in general.
If the single longest times blocked on a specific CS/LCS ("G" in the list above) is high that indicates that the CS/LCS in question may have causes the game to freeze for brief periods (ie stutter). If you suspect this to be the case you may try searching the log for that objects address, as in addition to the summaries the profiling stuff also prints lines about specific times Oblivion/Fallout blocked, though only if the blocking lasted for a while. Thus, you can find out how many times a critical section blocked for a long time by searching the log for those lines, and you can also find out which thread blocked on it (thread 1 blocking for more than 33 milliseconds will always result in a stutter at 30 FPS; other threads blocking for a long time will often cause problems, but not always, and it may take them longer before it qualifies as a problem). Currently those lines may misidentify LCSes as CSes, that will get fixed in the next version.
If a CS/LCS blocked many times, with a large total-times-blocked but a smaller longest-time-blocked then that CS/LCS is more likely to cause general performance reduction (ie reduced FPS) than actual stuttering.
The performance summaries printed are cumulative - a summary printed at the end of a hour long game session includes all blocking events for the entire hour, not just the recent ones.
If a CS/LCS caused problems for a smaller part of the time, but had no issues most of the time, then the information about its problems could get diluted by all the times it wasn't having problems. You can check for this by comparing multiple summaries, or by getting a log of a game session that included mostly just problem times.

Once a critical section is suspected of being a problem, what can be done about it?
Well, really, the critical sections largely reflect the structure of Oblivion/Fallout code, which (mostly) nothing can be done about. But... some things can be tried, and in a few cases they have been found to be quite helpful.
One thing that can be done is a critical section or light critical section can be suppressed. This forces the program to never block on that CS/LCS, usually causing the program to behave incorrectly and/or crash. In some cases however there seem to be little or no harmful consequences.
A less-likely-to-crash possibility is that the CS/LCS behavior can be tweaked. Its behavior can be modified to make it give priority to threads under specific circumstances, or to make it more willing to spend CPU time to reduce real time spent blocking. However, Stutter Remover is already applying a set of default tweaks (determined by CriticalSection\iDefaultMode and other such settings) to each CS & LCS, to get a noticable improvement requires that a set of alternate settings be not just noticably better than vanilla Oblivion/Fallout behavior but noticably better than default OSR/FSR behavior.

The "OverrideList" section lists specific objects that you want deviations from the default behavior on. For each override, you specify a type of object, a way of finding that object, and what behavior changes you want. Note that overrides of a specific type will be ignored if the appropriate bUseOverrides setting is 0.
For critical sections and light critical sections, the way of find it is either by object address or by caller address. All addresses must have an "0x" added at the beginning, otherwise Stutter Remover will try to interpret them as decimal and get very confused. If the CS/LCS is listed as "static" in the profiling summary, then object addresses should work fine, just set it to the object address the profiling stuff listed. If the CS/LCS is listed as dynamic instead, then the object address has a habit of changing with the phase of the moon or other arcane circumstances, so another method should be used (though you can often get away with testing a change using the object address of a dynamic object address, just be aware that it will probably silently fail when you switch computers, reboot, update drivers, or somesuch thing). Caller address is currently the only alternative. However, for performance reason caller address currently only works if the first caller is the one you specified. For most critical sections, the first one in the list never changes, so you can simply use that address, but for most LCSes and some rare CSes the order may change from run to run, in which case you may need to create multiple override entries for each possible first caller.
Once you've told it how to find a CS or LCS, then you tell it the behavior differences you want. Currently this means a line like "Spin = 500" or "Mode = 1".

These are the default overrides atm:
OverrideList = {    CriticalSection = {        CallerAddress = 0x828509        Mode = 5        comment = Renderer+0x180, recommendation=suppress (mode 5)    }    LightCriticalSection = {        CallerAddress = 0x86D7EA        Mode = 3        comment = MemoryPool (multiple LCSes), recommendation=stutter (mode 3)    }    LightCriticalSection = {        CallerAddress = 0x86D94C        Mode = 3        comment = MemoryPool alternate caller (multiple LCSes), recommendation=stutter (mode 3)    }    LightCriticalSection = {        ObjectAddress = 0x1079FE0        Mode = 2        comment = GarbageCollector, recommendation=fair (mode 2)    }}

The modes are:
1 = more or less vanilla behavior - often tends to give priority to threads trying to reenter shortly after exiting
2 = increased fairness (attempts to let threads in in the order that tried to enter)
3 = stutter (vary behavior once in a while)
5 = suppress (will generally crash, but when it works its the highest performance)
6 = attempts to give priority to the main thread
7 = attempts to reduce priority for the main thread
CSes and LCSes use different implementations of each mode, so the actual meanings may vary, but both attempt to use the same basic concept for a given mode number.
Other "Mode" values will tend to generate warnings or error messages and either have no effect or cause Oblivion/Fallout to exit immediately.
Spincounts:
low = minimize CPU resources used (in theory; practice differs)
high = minimize time blocked at the expensive of increased CPU usage (in theory; practice differs)
0 = very low
500 = low
1000 = medium-low
2000 = medium
4000 = high
In practice, I think "low" has tended to work better as the default setting.

Fallout3 CSes and LCSes with known names, purposes, or characteristics:
Renderer+0x180 CS: called from 0x828509; this CS seems to be okay to suppress; this was a major source of performance problems in Oblivion, slightly less so in Fallout.
Heap LCSes: These are dynamic LCSes called from 86D7EA or 86D94C. They are created and used inside of Fallouts heap, and become irrelevant if the heap is replaced. I think the equivalent CS in Oblivion worked best when set to stuttering, so that's what I've set these to in the default ini of FSR for the moment.
GarbageCollector LCS: static, object address is 0x1079FE0
ExtraData LCS: static, object address is 0x0106C980
TESObjectCell::CellRefLockEnter LCS: static, object address is 0x00DCD0A8
GarbageCollector LCS: static, object address is 0x01079FE0
NavMeshObstacleManager+0x100: dynamic, called from 0x005E920D among other addresses
User avatar
john page
 
Posts: 3401
Joined: Thu May 31, 2007 10:52 pm

Post » Mon Feb 01, 2010 10:35 am

Skyranger, there seems to be a CTD caused by the latest alpha on Nexus. I've been running the beta and some previous alpha versions without a hitch, this issue just popped up with the last one.

There's also an interesting difference in these CTDs, which made it actually possible for me to determine that it was caused by the alpha version:
Normally Fallout 3 will crash on rare occasions. An error message will pop up "Fallout3 stopped working...etc", standard stuff. However in this case Fallout would instantly close, without any error message or anything to that effect.
Interesting enough, these are actually the kind of crashes I normally only get if I don't add the "inumhwthread=2" cpu core limitation to the fallout.ini

I'm back on the beta now which works without a hitch.
User avatar
Eve(G)
 
Posts: 3546
Joined: Tue Oct 23, 2007 11:45 am

Post » Mon Feb 01, 2010 2:03 pm

The latest alpha seems stable to me, I let it write it own .ini instead of using the one supplied, though I dunno theres any difference.
User avatar
Lisha Boo
 
Posts: 3378
Joined: Fri Aug 18, 2006 2:56 pm

Post » Mon Feb 01, 2010 6:34 am

Just dropping in to say thanks for FSR, both for Oblivion and FO3. Great work! :)
User avatar
lucy chadwick
 
Posts: 3412
Joined: Mon Jul 10, 2006 2:43 am

Post » Mon Feb 01, 2010 4:16 am

Skyranger, there seems to be a CTD caused by the latest alpha on Nexus. I've been running the beta and some previous alpha versions without a hitch, this issue just popped up with the last one.

There's also an interesting difference in these CTDs, which made it actually possible for me to determine that it was caused by the alpha version:
Normally Fallout 3 will crash on rare occasions. An error message will pop up "Fallout3 stopped working...etc", standard stuff. However in this case Fallout would instantly close, without any error message or anything to that effect.
Interesting enough, these are actually the kind of crashes I normally only get if I don't add the "inumhwthread=2" cpu core limitation to the fallout.ini

I'm back on the beta now which works without a hitch.
The LCS full hook code is known to cause crash-on-exits, and suspected of causing other issues. You might try setting LightCriticalSections\bFullHooks to 0, that disables the most suspect LCS code. Unfortunately, that includes code necessary for using LCS overrides and some of the smarter LCS modes.
User avatar
Ice Fire
 
Posts: 3394
Joined: Fri Nov 16, 2007 3:27 am

Post » Mon Feb 01, 2010 2:57 pm

The LCS full hook code is known to cause crash-on-exits, and suspected of causing other issues. You might try setting LightCriticalSections\bFullHooks to 0, that disables the most suspect LCS code. Unfortunately, that includes code necessary for using LCS overrides and some of the smarter LCS modes.

I'll try the alpha again and disable that hook, see if that stops the CTD.
User avatar
Katy Hogben
 
Posts: 3457
Joined: Mon Oct 30, 2006 12:20 am

Post » Mon Feb 01, 2010 4:45 am

I am impressed! Thank you very much!

I'm using the "old" alpha with MaximumFPS set to 60 and FO3 runs really, really smooth.
Unfortunately the "new" alpha CTDs without error message for me as well.
User avatar
Fanny Rouyé
 
Posts: 3316
Joined: Sun Mar 25, 2007 9:47 am

Post » Mon Feb 01, 2010 2:51 pm

There is an issue with the hashtables resizing code that can conflict with FOSE. I believe that hashtable resizing was disabled by default in the most recent alpha, so you only really need to know that if you enabled it manually.

My current best guess is that better hashtable sizes grants a significant benefit to Oblivion, but only a little benefit to Fallout. So, I'm going to try to get it working right on Oblivion, but if its something painful to port to Fallout then the feature may be dropped from Fallout versions of Stutter Remover.
User avatar
Christina Trayler
 
Posts: 3434
Joined: Tue Nov 07, 2006 3:27 am

Post » Mon Feb 01, 2010 11:22 am

There is a new extra-alpha-ish version up at:
ftp://71.115.222.171/sr_Fallout_Stutter_Remover.dll

The old alpha is temporarily still available:
ftp://71.115.222.171/old_alpha/sr_Fallout_Stutter_Remover.dll


I just downloaded the old alpha from the second link. The instructions at the nexus site say to "Drag the "Data" folder from the zip to your Fallout folder", but there is no zip nor data folder, only a .dll file (sr_Fallout_Stutter_Remover.dll).

Did I download the correct file, and if so, where should I put it?

Thanks.
User avatar
Quick Draw III
 
Posts: 3372
Joined: Sat Oct 20, 2007 6:27 am

Post » Mon Feb 01, 2010 2:08 am

I just downloaded the old alpha from the second link. The instructions at the nexus site say to "Drag the "Data" folder from the zip to your Fallout folder", but there is no zip nor data folder, only a .dll file (sr_Fallout_Stutter_Remover.dll).

Did I download the correct file, and if so, where should I put it?

Thanks.


Put the .dll file into the Fallout 3\Data\FOSE\plugins folder.
User avatar
Dan Scott
 
Posts: 3373
Joined: Sun Nov 11, 2007 3:45 am

Post » Mon Feb 01, 2010 3:16 am

Great, thanks.
User avatar
phil walsh
 
Posts: 3317
Joined: Wed May 16, 2007 8:46 pm

Post » Mon Feb 01, 2010 5:56 am

Should I enable/force vsync when using this mod, or is it better to force off via the .ini and NV control panel?
User avatar
Marcin Tomkow
 
Posts: 3399
Joined: Sun Aug 05, 2007 12:31 pm

Post » Mon Feb 01, 2010 9:37 am

Ok I'm an idoit. I have no idea how to install FOSE. There doesn't seem to be any documentation anywhere. I downloaded 7zip but now what?

Edit: N/M i got it. Sorry.
User avatar
Marion Geneste
 
Posts: 3566
Joined: Fri Mar 30, 2007 9:21 pm

Post » Mon Feb 01, 2010 5:27 am

Ok I installed FOSE but I dont have a "Fallout 3\Data\FOSE\plugins folder"
User avatar
phillip crookes
 
Posts: 3420
Joined: Wed Jun 27, 2007 1:39 pm

Post » Mon Feb 01, 2010 9:35 am

Ugh. This is starting to drive me nuts. Do I need this GECK thing now too? There's no "plugins" folder anywhere in my Fallout directory.
User avatar
mollypop
 
Posts: 3420
Joined: Fri Jan 05, 2007 1:47 am

Post » Mon Feb 01, 2010 1:10 pm

Ugh. This is starting to drive me nuts. Do I need this GECK thing now too? There's no "plugins" folder anywhere in my Fallout directory.

Right click, create new folder..
User avatar
Ron
 
Posts: 3408
Joined: Tue Jan 16, 2007 4:34 am

Post » Mon Feb 01, 2010 3:41 pm

Probem is there's no FOSE folder either. Am I supposed to create it also?
User avatar
Melly Angelic
 
Posts: 3461
Joined: Wed Aug 15, 2007 7:58 am

PreviousNext

Return to Fallout 3