Oblivion Stutter Remover
version 3 beta 6
by SkyRanger-1

Forum thread: http://www.gamesas.com/bgsforums/index.php?showtopic=968943
TESnexus page: http://tesnexus.com/downloads/file.php?id=23208

This is an OBSE plugin, and it will not work without a recent version of OBSE.


====================================
0. Contents:
====================================

0. Contents
1. Overview
2. Installing
3. Uninstalling
4. Settings
5. Known Problems
6. How This Works
7. Bug Reporting
8. Credits


====================================
1. Overview:
====================================

This plugin makes Oblivion not "stutter" as much, and generally feel smoother. It prevents or mitigates a number of issues related to stuttering and framerates, including reducing the frequency of stutter related crashes. For more detail, see Section 6: How This Works.

Note however that this is not a substitute for Silent Feet / Operation Optimization / Oblivion Crash Prevention System / etc. For instance if you get major stuttering because of sound driver or sound hardware issues, this could help some but Silent Feet would probably help more.

This is compatible with everything except earlier versions of itself. The only caveat is that Streamline (and other mods that monitor FPS) will not be able to accurately measure FPSes outside of the target range set by this plugin (15 to 30 by default), so Streamlines FPS targets should lie between this plugins FPS targets.


====================================
2. Installing:
====================================

The installation process is:
1. If you were using version 2 of this plugin, uninstall that first (ie delete or move FPS_Capper.dll).

2. Simply drag the "Data" folder from the zip to your Oblivion folder.
If you are installing a version of this that does not come in a zip file and/or does not have a "Data" folder in it, instead place the file sr_Oblivion_Stutter_Remover.dll in to your Oblivion\Data\obse\plugins folder (if you don't have such a folder, create it).

3. [Optional] If you had used an earlier variant of version 3 of this plugin, consider deleting your sr_Oblivion_Stutter_Remover.ini file. It is in the Oblvion\Data\obse\plugins directory. Deleting it will cause this plugin to create a new ini file with the current default settings the next time you run Oblivion. Since one of the ways this plugin improves between versions is the addition of new settings and the tweaking of old defaults, using new defaults could be better than using your old settings.


====================================
3. Uninstalling:
====================================

Simply delete the sr_Oblivion_Stutter_Remover.dll file from your Data\obse\plugins folder.


====================================
4. Settings:
====================================

In general, this attempts to have decent default settings so that users are not required to monkey with them.

However, there are a few cases where it might be desirable or even important for you to tweak your settings. In particular, if your refresh rate in the graphics mode that Oblivion runs in is something other than 60 hertz, several minor settings will be sub-optimal for you. If you don't know what your refresh rate is in Oblivion, just assume it's 60 hertz - this is by far the most common refresh rate.

At the moment, the most important tunable setting is iHeapMode. It defaults to a value of 0, which disables the heap-replacement code in this plugin. Set to 1,2,3,4, or 5, it will replace Oblivions (poorly performing) heap manager with a different one, which can noticably improve performance. Unfortunately, this tends to cause instability for many users (reason unknown at time). Still, many people report that they can get Oblivion to be stable with at least one of the alternative heap managers this plugin offers. Heap manager #4 should not be used, and heap manager #2 should not be used on XP. Of the rest, #1 tends to have the highest performance.

These are the settings and their current default values (may not be 100% up to date):

[General]
MinimumFPS=10
MinimumFPS should be a framerate lower than you normally get will playing oblivion, and higher than unplayable framerates. I currently recommend 10. If Oblivion tries to drop below this framerate for even a single frame then this plugin will force Oblivion in to slow motion until it rises above this framerate. Note that this setting is ignored when in menus in recent versions - it now only effects game mode. You can set this to 0 to disable this feature.

MaximumFPS=30
MaximumFPS should be a framerate high enough to appear smooth. I currently recommend 30. It's somewhat important for this to be an integer divisor of your refresh rate (ie on 60 hertz that means 60, 30, or 20). If Oblivion tries to rise above this framerate for even a single frame then this plugin will force it to slow down by putting Oblivions main thread to sleep and/or burning CPU cycles in a loop that does nothing.

iSleepExtra=5
If this number is zero or greater, then this is a number of milliseconds that Oblivion should sleep each second. This means that increasing this number will reduce your framerate. However, it has been found that increasing this number prevents some types of stuttering for some users, and is very much worth the too-small-to-measure impact on framerate. But, this option may become mostly obsolete due to threading-related improvements that appear to be able to provide the same benefit without hurting FPS at all, at least on multicore CPUs.
If this parameter is -1 then it puts this mod in to a sort of test mode where it will not put Oblivions main thread to Sleep for any reason.

bFPSConsoleSPAM=0
If this option is not zero the this plugin will print performance data to the console EVERY SINGLE FRAME.

iSchedulingParanoia=2
This setting is only meaningful if MaximumFPS is greater than zero. Setting this option to 0 helps this plugin to find more spare milliseconds that it can make Oblivion sleep. Setting it to the same value as iSchedulingResolution will make it harder for this plugin to find spare milliseconds, but will also make it less likely for the MaximumFPS stuff to accidentally reduce the framerate by the wrong amount. Setting it to twice the value of iSchedulingResolution will make it harder still to find spare milliseconds, and less likely for MaximumFPS to to be off. This option is in units of milliseconds.

iSchedulingResolution=1
This sets Oblivion to a higher resolution scheduling mode. It is recommended that this be 1 for the vast majority of users - otherwise MaximumFPS and iSleepTime won't work as well. This seems to be responsible for some of the stability improvements this plugin produces. You can set this to 0 to disable this feature, or to 2 or 5 to turn it down somewhat.

iHeapMode=0
This setting controls how Oblivions memory manager (aka heap, aka heap manager) works. This setting can significantly improve loading time, stutter, and/or FPS. The details aren't well known yet, but reports on performance are positive. Unfortunately, the current implementation doesn't work well on some computers, resulting in instability. At this point the suspicion is that some alternate heaps are exposing bugs in Oblivion that were hidden by its vanilla heap. The recommended setting for stability atm is 0, the recommended setting for performance is 1.
0: vanilla Oblivion heap
1: FastMM4 heap, fast, requires Oblivion\Data\OBSE\plugins\ComponentDLLs\BorlndMM.dll
2: Windows heap, slow on WinXP, decent on Vista
3: a simple heap I wrote for this, probably not as good as FastMM4 but doesn't require a seperate dll
4: FastMM4 debug heap, requires Oblivion\Data\OBSE\plugins\ComponentDLLs\debugMM.dll (do not use this setting)
5: another heap I wrote for this, a bit faster than #3 above

iHookMode=0
The meaning of this setting may change.

[CriticalSections]
iCriticalSectionMode=2
Oblivion uses a number of CRITICAL_SECTION objects, which are a microsoft implementation of a type of thread synchronization object. This setting determines how this plugin changes them:
0: no changes, use vanilla critical sections
1: override vanilla spincounts (reduces stuttering for some users)
2: DEFAULT: override vanilla spincounts & optimize for "fairness" (reduces stuttering more)
3: debugging mode: displays performance data to console

iCriticalSectionSupression=1
This option lets you trade stability for a boost in performance (reduction in stuttering). By default, it is set to a very small loss in stability for a very large gain in performance. This option is a bitmask: you can add together different flags to get the combination of them.
0: maximum stability (no critical sections are suppressed)
1: RECOMMENDED: huge reduction in stuttering; no stability reduction has been observed, but theoretically some is anticipated (critical section Renderer+0x180 is suppressed)
2: some reduction in stuttering; some people get crashes on interior->exterior transitions with this enabled (critical section Renderer+0x080 is suppressed)
4: NOT RECOMMENDED: causes massive instability; probably improves load times, FPS, and stuttering; suppresses heap critical section, has no effect if Oblivions heap is overriden via iHeapMode. (memory heap critical section @ 0xB32B80 disabled)
8: Probably no significant performance effects. Untested. (ExtraData critical section @ 0xB33800 is disabled)
16: Theoretically, probably causes massive instability and a small improvement to both FPS and stutter. Untested. (Unknown4 critical section @ 0xB3FA00 is disabled)
32: Theoretically, probably causes stutter to be reduced slightly when moving at very high speeds in exterior cells for an extended period of time, for a small cost in stability. Untested. (Unknown6 critical section @ 0xB3FC00 is disabled)

iDefaultSpin1=500
iDefaultSpin2=500
iMemorySpin=-1
iExtraDataSpin=-1
iUnknownSpin1=-1
iUnknownSpin2=-1
iUnknownSpin3=-1
iUnknownSpin4=-1
iUnknownSpin5=-1
iUnknownSpin6=-1
iSpecialSpin1=2000
iSpecialSpin2=1000
iSpecialSpin3=4000
These settings are used to adjust the spincounts of Oblivions critical sections. In this context a spincount is an amount of time that Oblivion will wait for a resource before blocking (entering in to an alternate state in which it stops consuming resources while waiting; entering and exitting this alternate state is slow however). They are in units of time, but the units are arbitrary and unspecified and vary depending upon CPU model, though very short (perhaps 50 nanoseconds or so). These settings only effect computers with some form of hardware multithreading (meaning multicore CPUs or hyperthreading CPUs or multiple CPUs).


====================================
5. Known problems:
====================================

Currently, the most stable version is considered to be version 3 beta 5.

version 3 beta 6: None yet.
version 3 beta 5: Reports of very rare drastic FPS drops, fixable by restarting Oblivion.
version 3 beta 4: Don't use this version! In some cases nearby NPCs would randomly die whenever you did a cell transition.
version 3 beta 3: NPC face movements can screw up in dialogue, but not nearly as bad as in beta 2.
version 3 beta 2: NPC voice and face movements can screw up in dialogue.
version 3 beta 1: Sometimes freezes for several minutes at the main menu. Also, NPC voice and face movements screw up in dialogue.

note that this only includes unintended behaviors that occur on the default settings


====================================
6. How This Works:
====================================

This is an OBSE plugin dll. It basically hacks Oblivion.

6.1: General stuff:
The "main hook" of this plugin monitors framerates and adjusts the flow of gametime. It mitigates stuttering by making Oblivion game logic not skip ahead when it does stutter. Effectively, frames that take a long time end up being in slow motion. This is done by making Oblivion act as if iFPSClamp were set to MinimumFPS, but only for frames that are slower than MinimumFPS. This is believed to also improve stability. It can also impose a maximum framerate - some people percieve Oblivion as smoother when its framerate is prevented from exceeding half the refresh rate, plus this helps free up resources for Oblivions secondary threads.

The main hook also puts the main thread of Oblivion to sleep for brief periods of time, which has been oberved to improve stutter for some people (though that functionality is now believed to be made redundant by other things this plugin does and may be removed).
The main hook also handles a variety of initialization tasks for this plugin. The initialization code can also switch Windows in to a higher resolution scheduling mode while Oblivion is running, which helps with a few things.


6.2: Critical Section stuff:
Critical sections are microsoft-provided thread synchronization primitives that Oblivion uses internally to make sure that threads don't accidentally corrupt each other. This plugin modifies Oblivions critical sections to use different spincounts (a tunable parameter of CRITICAL_SECTION objects), controlled by the various "Spin" settings. This plugin also modifies the implementation of CRITICAL_SECTION objects from the normal microsoft implementation slightly to make them more "fair" (they will sacrifice a little performance to make it less likely for a single thread to end up hogging a resource that other threads want for a long time). That part is controlled by the iCriticalSectionMode setting. There is an additional tweak specifically for the critical section used by Oblivions normal heap manager, controlled by the setting iMemoryUnfairness.

This plugin can also suppress specific critical sections. This is controlled by the iCriticalSectionSupression setting. Setting it to 0 causes no critical section to be suppressed. Setting it to 1 causes a single specific critical section to be suppressed, chosen because it seems to both be unnecessary and the cause of a large fraction of all stuttering that Oblivion does. The default / recommended setting is 1. Other values can suppress other critical sections, or suppress multiple critical sections. Typically, suppressing a random critical section will cause a miniscule improvement in performance and a large reduction in stability, but certain critical sections offer a more favorable tradeoff.

6.3: Heap Replacement:
Oblivion uses a custom heap (aka memory manager aka malloc/free) implementation that appears to have been written by Bethesda specifically for Oblivion. Their implementation is flawed. This plugin now has the ability to replace the Oblivion heap manager with a variety of alternatives, most of which are MUCH faster. Unfortunately, changing the heap implementation seems to cause Oblivion to become unstable for many users. I'm not certain if this is due to a flaw in my method of replacing the heap, or if this is due to bugs in Oblivion that happen to not crash with Oblivions vanilla heap. I'm leaning towards the later at the moment.

The performance improvement offered by this is large (reduces stutter, reduces load times, makes certain menus faster, may improve FPS slightly). Unfortunately, because it has a tendency to produce instability I have set this feature to default to disabled for the time being. However, many users consider it worthwhile to experiment with the various settings to find one that is stable and fast on their particular Oblivion installation. This is controlled by the iHeapMode setting.


====================================
7. Bug Reporting:
====================================

Report bugs to the forum thread linked at the top of this file. The following is information is likely to be important:
Symptoms of the bug
Circumstances where you see the bug
Version of this plugin that you are using
Contents of your sr_Oblivion_Stutter_Remover.ini file
Anything you think might be important.

For certain bugs, additonal information might be relevant:
What CPU you are using. If your CPU supports hyper-threading, whether or not hyper-threading is enabled.
What OS you are using.
Any distant LOD mods you are using.
What graphics card or DirectX version you are using.


====================================
8. Credits:
====================================

This plugin was made by me (Christopher Doty-Humphrey).

It would not have been possible without the enormous efforts made by the OBSE team. In addition, Ian Patterson (an OBSE team guy) has helped me through a lot of spots where I had trouble.

The original thread that prompted me to start this plugin was started by DeviusCreed.

Numerous testers have supplied useful feedback. In particular mashani's information about which settings produced which results for him helped me understand why early versions of this were producing unexpected benefits, and led to a number of the features and settings in later versions.

The following tools were used in the production of this plugin:
Oblivion, by Bethesda
OBSE and the OBSE source code
Microsoft Visual C++ 2008 Express Edition
IDA Free (Interactive Debugger, free version, version 4.9)
Cheat Engine (version 5.4)
Plus the obvious stuff like Windows XP, Notepad, and Firefox. Note that all of those (except Oblivion itself) are (legally) available for free.

With the exception of Oblivion and Windows XP, all of those are available free of charge.

I've also had Hex Workshop and ollydbg recommended to me but haven't gotten around to trying them yet.

I don't know if it's already been reported, but heap 5 caused me to lose my hit sounds in combat.

This mod is absolutely fantastic. Thank you very much!

But I have a small question:

I use Alternatives to death and reload from kuertee and it seems to me after a few tests that the scripts from his mod are not executed when Oblivion Stutter remover is installed. I made several tests and without stutter remover his mod works and with
Stutter remover it doesn't work.

Picture of console output: http://img5.imagebanana.com/view/gh1afvsr/...91113161803.jpg


I am quite sure that this can be fixed with ini settings but I do not understand how this mod works so I have to ask here.

Well, I can confirm that it's reproducing for me since v265, when NIFZ and KFFZ started using MelStrings. Just to clarify, the issue is that the list of zstrings should have an extra zero as the list terminator, as per http://www.uesp.net/wiki/Tes4Mod:Mod_File_Format/CREA.

The easiest way to trigger is using the Stutter Removal mod with Borland heap (iHeapMode=1) - seems tied to the way small block allocator handles zero byte allocations. The crash occurs early during Bashed Patch esp load, while overriding one of the CREA records added by MMM; stack trace points right on NIFZ string loading code, with a bogus start address. It disappeared once the eol byte has been added in dumpData().

I've been tracking an issue with heap 1 that I can now reproduce using the lastest version of OSR. Basically, I see white pixel boxes when summoning/unsummoning (occurs for both PC and NPCs) as well as with spectral creatures from OOO after about 5 to 10 minutes of play. The original thread is here (screenshots attached). An even older thread documenting the white pixel boxes issue when summoning/unsummoning with previous versions of OSR is here (screenshots also attached).

I also tested with heap 3 and 5. Heap 5 does not cause the white pixel boxes but causes CTD's after 5 or 6 cell transitions. Heap 3 seems to be the most stable on my setup. However, I am experiencing an issue where sound is "cut off" in certain dungeons (I do not experience that with heap 5).

Yeah, Heap mode 1 is pretty good in case of performance, but it gives me strange LOD problems after minutes of play, as you can see in the posted picture:

[IMG]http://s1.directupload.net/images/090924/temp/wc979n4o.jpg[/IMG]

LOD textures ain't getting removed from some building in cities, the houses are still there, but behind those LOD objects. This is really annoying.

Any help for this problem?

That sounds fixable. I can certainly alter the behavior for 1 specific allocation size. But that can't be the only issue - my computer works fine with some heap modes, but heap #1 crashes for me and I don't use a Bashed Patch.

Yes, it's fixable. Someone posted a python code snippet that made it work for me when I tried it. So it can be easily dealt with in Bash.

I tried it with Wrye's code snippit and it worked great. I can now use HeapMode 1. And it's quite nice so far I must say.

Where can I find this Wrye's code snippit?

A better solution is probably:

	def dumpData(self,record,out):		"""Dumps data from record to outstream."""		strings = record.__getattribute__(self.attr)		if strings:			out.packSub0(self.subType,null1.join(strings)+null1)

See if that works. Should prevent unnecessary writing of the sub-record.

Next question:
1. Where can I find this bosh.py file? I found 2 bosh files that do not open except for less than second and close themselves - they are in Program Files...\ Oblivion\Mopy but probably I need something else?

2 Will this be addressed in future Wrye bash updates?

I love the person who found this dearly now

O...K...
Found file, opened it. Wrote on 1022 what Wrye advised. 8 times. Every time with typing mistake.
Can one send me the fixed file so that I just drop it in my Mopy folder and stop pretending I know what I am doing? e-mail: svilenialbena@aol.com
Thanks

Heres the edited file http://www.filefront.com/14539325/bosh.py/

Gee thanks, I'm hoping you mean that in a heterosixual way ... just got lucky stumbling over the root cause.

I'm hoping to pull together a larger contribution in the foreseeable future - a savegame cleaner that works quite nicely in conjunction with OSR. It may take a while though, but still working on it.