Repairing the Cogs of Morrowind No. 20 : III - Morrowind

» Mon Nov 22, 2010 8:20 am

Cryengine 3

Slower than anything in recorded history.

Unreal 3

One of the worst render engines of the 21st century. That, or has been used by a record number of games with the programmers making the textures. Specular should never have been allowed.

Stalker

Fast on low settings, slow when it looks good. The best example out of the list, though.

3 of them offer day night cycles, all are using some kind of deferred rendering approach. This will take the 6-8 light limit and we can light the distant land too. But this is probably too experimental.

Deferred lighting is appropriate in a few situations, mainly when you absolutely have to have more than 8 lights (you can also do multi-pass rendering and not worry about deferring the process) or need specialized effects beyond what traditional lighting can do. If you were to implement full global illumination, true ambient occlusion and maybe subsurface scattering, the the work to do deferred rendering would be worth it. With what Morrowind has, there's just no point. Adding more than 8 lights would also take scripted management of some kind, which would be a right pain to add. Either way, this would be easier to do in MGE where there isn't a code-size limit and it doesn't have to be written in assembly.
And distant land can already be lit, without terribly much work actually. A per-vertex lighting shader is easy to write and could be added to ingame.fx, if it's not in there.

» Mon Nov 22, 2010 1:15 am

Just saw the new images of the shadows http://www.tesnexus.com/downloads/file.php?id=26348, looks awesome. How's the performance/stability?

That is incredible. So, there IS a performance impact correct?

Will it ever be possible to have shadows with good performance?

» Mon Nov 22, 2010 2:35 am

Slower than anything in recorded history.

Come on PK, Cryengine 3 is not out yet. Have a little faith in them. They made it work on consoles, that should count.

One of the worst render engines of the 21st century. That, or has been used by a record number of games with the programmers making the textures. Specular should never have been allowed.

Unreal makes use of material shaders which need combined skills of a programmer and texturer.

One of the most successful engines of 21st century.

Deferred lighting is appropriate in a few situations, mainly when you absolutely have to have more than 8 lights (you can also do multi-pass rendering and not worry about deferring the process) or need specialized effects beyond what traditional lighting can do. If you were to implement full global illumination, true ambient occlusion and maybe subsurface scattering, the the work to do deferred rendering would be worth it. With what Morrowind has, there's just no point. Adding more than 8 lights would also take scripted management of some kind, which would be a right pain to add. Either way, this would be easier to do in MGE where there isn't a code-size limit and it doesn't have to be written in assembly.
And distant land can already be lit, without terribly much work actually. A per-vertex lighting shader is easy to write and could be added to ingame.fx, if it's not in there.

"With what Morrowind has..."
"With what Morrowind has..."
(Dramatic Serenity speech)
2002 version or 2010 version.

Addition of shadows already means some kind of combination of MGE and Morrowind renderers (as seen in the screens). My initial theory was always about merging them and the work would be done in MGE.

Peace! Thanks for all the hard work our beloved programmers. I am just stinging you with questions as a gadfly like Socrates. :frog: I hope I won't end up with being executed. :angel:

» Mon Nov 22, 2010 5:34 am

Ahaha you got there eventually. It's a custom port of MGE (to gcc), which began with noticing how terrible bad the MGE source tree was; it uses the distant land objects as shadow casters. I've made it a bit more efficient in the process, but only a little bit. Shadows run pretty fast by themselves, so don't worry. I had to cleave off MWSE, since the integration was pretty ugly, and the VC specific inline assembler.

But my computer can't run Crysis, GF 7600GS AGP style, so it's going to be a while.

» Sun Nov 21, 2010 7:10 pm

Ahaha you got there eventually. It's a custom port of MGE (to gcc), which began with noticing how terrible bad the MGE source tree was; it uses the distant land objects as shadow casters. I've made it a bit more efficient in the process, but only a little bit. Shadows run pretty fast by themselves, so don't worry. I had to cleave off MWSE, since the integration was pretty ugly, and the VC specific inline assembler.

But my computer can't run Crysis, GF 7600GS AGP style, so it's going to be a while.

So, you can actually have good performance while running those beautiful shadows?

Can these shadows be done for Oblivion eventually?

» Sun Nov 21, 2010 9:49 pm

This sounds really good. Any chance you′d reintegrate MWSE, but better=

» Sun Nov 21, 2010 10:31 pm

Hi Hrnchamd - minor request or at least a question - I'm about to finally upgrade to a Win 7 64bit - as the system I have will use 4gb of ram I've been looking at a patch for Morrowind called the 4GB_patch file exe which allows the game to access more memory.

I've put the readme for the patch under this spoiler

Spoiler

Before I started, I tried one last thing. I remember someone posted here a while back about having a mod-heavy Morrowind installation and using a generic EXE patcher to allow Morrowind to address 4 GB of memory instead of 2. Apparently that solved all sorts of crashing problems for this person IIRC. So, I Googled it and tried it. Bingo. My completely repeatable "Loading Area" problem vanished. I loaded my Pelagiad save, Recalled, and then started casting Divine Int and Almsivi Int and hopping around from place to place. Perfectly stable, no crashes.

More testing is in order, but it looks like I found my culprit. This is the one and only thing I've tried which has budged the problem even a little. Note that none of this was necessary in Vista -- everything just worked. Something in the executable's use of memory has changed from one O/S to the other, generating massive instability without the 4 GB patch in place. It does seem to involve heavy use of mods, too -- the virgin installation seemed fine with an unpatched EXE.

In any case, here's the link, in case anyone else finds themselves in the same situation.

I originally wrote this tool for a friend of mine who needed it. This very little tool patches x86 executables in order to let them have 4GB (instead of only 2) of virtual memory on x64 platforms. This tool comes very handy for applications which need a great amount of virtual memory like games, 3D renderization, multimedia etc. To gain these 2GB, you just have to use this tool to patch the executable (*.exe file) of the software you want to have these additional GBs of virtual memory. It can be used by clicking on it and choosing the file or through command line (e.g.: "4gb_patch file.exe"). It automatically creates a backup copy of the original executable.

Why things are this way on x64 is easy to explain. On x86 applications have 2GB of virtual memory out of 4GB (the other 2GB are reserved for the system). On x64 these two other GB can now be accessed by 32bit applications. In order to achieve this, a flag has to be set in the file's internal format. This is, of course, very easy for insiders who do it every day with the CFF Explorer. This tool was written because not everybody is an insider, and most probably a lot of people don't even know that this can be achieved. Even I wouldn't have written this tool if someone didn't explicitly ask me to.

My question is - if I first patch my MW exe with this and then apply MCP - can you envisage any disaster occuring?

My request is that if possible would you consider incorporating this type of patch into a version of MCP? That is if it is actually worth doing.

» Mon Nov 22, 2010 7:09 am

^ ^ ^ ^

I have wanted something like that for some time, and have debated that patch myself (never tried it yet though). I could certainly see that cutting down some performance issues and bottlenecks, and if it could be added to MCP, that would just make it even better (as I trust the testing done for MCP a bit more since I can actually see how seriously and thoroughly it is tested).

» Mon Nov 22, 2010 8:32 am

When I first posted information about the 4gb patch (I don't recall anyone mentioning it before that time, and I believe the above quoted person was referencing my posts), it was in regards to MGE sending Morrowind over the memory limit for x86 processes. From what I understand this is something simple that could be easily included in the MCP -- if Hrnchamd sees it inside the scope of his project.

For what its worth, the generic patcher has allowed me stability and has had no adverse affects. My patch order is always GOTY -> MCP -> 4gb, but I can't say for sure if the order is important.

» Sun Nov 21, 2010 8:00 pm

You should always apply the 4GB patch after each install of MCP. MCP has to rewrite the exe with the changes you requested, so other unofficial patches are erased as well.

People claim a lot of things fix crashes when it moves memory around enough to not hit the trouble spot, a problem with observer bias. For example, Exe Optimizer just changes some of the mathematical code to be more efficient, but most posts about it primarily state it prevents crashes, when that is an unlikely outcome for what it is doing. I expect the same with the 4GB patch, unless you have very heavy resource use.

I can't support a 4GB patch, as I don't have an x64 (or a good graphics card) to test with, but it doesn't seem harmful in any way.

» Mon Nov 22, 2010 4:21 am

You should always apply the 4GB patch after each install of MCP. MCP has to rewrite the exe with the changes you requested, so other unofficial patches are erased as well.

People claim a lot of things fix crashes when it moves memory around enough to not hit the trouble spot, a problem with observer bias. For example, Exe Optimizer just changes some of the mathematical code to be more efficient, but most posts about it primarily state it prevents crashes, when that is an unlikely outcome for what it is doing. I expect the same with the 4GB patch, unless you have very heavy resource use.

I can't support a 4GB patch, as I don't have an x64 (or a good graphics card) to test with, but it doesn't seem harmful in any way.

I agree that "moving things around" might just hide the trouble spot for many issues, but I'm as certain as I can be that the 4gb patch did what it was supposed to (in my case). With specific MGE settings (using up enough memory) I could guarantee a crash upon load and log the memory usage to confirm that it was going over the x86 limit.

I'll keep a mental note about what you said for MCP rewriting the EXE, thanks =).

» Mon Nov 22, 2010 10:12 am

I can't support a 4GB patch, as I don't have an x64 (or a good graphics card) to test with, but it doesn't seem harmful in any way.

Any millionaires out there want to volunteer to buy Hrnchamd a new computer? :hubbahubba:

» Sun Nov 21, 2010 10:24 pm

You should always apply the 4GB patch after each install of MCP. MCP has to rewrite the exe with the changes you requested, so other unofficial patches are erased as well.*snip*

I can't support a 4GB patch, as I don't have an x64 (or a good graphics card) to test with, but it doesn't seem harmful in any way.

Oh I can understand about not being able to support it then

I was unsure if I applied it after the MCP install whether it might rewrite any changes you had made but it sounds like that's the only way to do so, so I guess I can only test and see

Thanks for the quick reply

» Mon Nov 22, 2010 12:14 pm

It seems I have found something about fixed magnitude spells. A spell of 20-20 frost damage does 19 points of damage to the player (38 with 100% weakness to frost), 21-21 gives 20 points of damage.

» Mon Nov 22, 2010 3:14 am

Are those cast on self spells? There were a lot of random generators in the code, if you could narrow down the spell conditions, it would help.

» Sun Nov 21, 2010 9:59 pm

my two pence about the spell magnitude:

Spoiler

i don't think this is easy to change, at least from my point of view. i guess the game handles most values as floats (or for non-computer scientists: a 20 is actually something between 19.99999999999 and 20.000000000001 in memory) because they use less memory and are easier to handle. therefore a spell with 21-21 magnitude would float around somewhere between 20.00...01 and 20.99...9 but will never be exactly 21 aswell as a spell magnitude of 1-21 will float around 2 and 20. to rteplace them by straight integers - if even possible - could cause a performance hit. and to add a static +1 could interfere with random magnitude spells (the 1-21 magnitude spell example would cause damage between 3 and 21). the only solution i see would be to alter the way the game truncates/rounds the decimal places.
but... nevermind. just thinking loud ^^

one thing i found:
i don't know if this issue is already fixed, since the last version of MCP i used was 1.6 iirc.
what i know is that this is not related to the "mercantile bug" or it's fix. iirc this fix even boosts the effect.

if a merchant has a low disposition, you can outsmart them by adding stackable low value items (1 or 2 gold worth) in the trade window piece by piece.
after that you just have to remove the whole stack of your items at once back to your inventory.
the result is having the difference between prices caused by the traders disposition as a surplus. and the trader gladly accepts the price even if you just sell him one of those items.
i think this is due to the game cannot lower the sale value of items worth only 1 or 2 gold but calculates the sale value of the whole stack, when removed

otherwise if you try this trick on a merchant with a high disposition you end up with debts.

i don't know if this can be fixed. and i don't know if i would be that happy if it IS fixed (since i always enjoyed selling Arrille a glass bottle for 800 gold for some extra starting cash :celebration: ).
i never considered myself as a cheater and always thougth that my character was a bit smarter (and better in maths) as the merchants.

» Sun Nov 21, 2010 10:03 pm

Are those cast on self spells? There were a lot of random generators in the code, if you could narrow down the spell conditions, it would help.

No, this spell wast cast by an NPC triggered by dialogue (cast "..." player). I made a little mod to test how "weakness to" spells stack and saw the magnitude thing.

» Sun Nov 21, 2010 8:38 pm

You have done a great work on this Hrnchamd. Thanks.

When people have too many savegames in their Saves directory, they start getting mysterious memory errors, the most commonly reported is:
"QuestName info in Journal "XXX" has no name string."

I did a little http://www.gamesas.com/index.php?/topic/893198-strangest-most-frustrated-problem-error/page__view__findpost__p__13093826 some time ago, and it's pretty reliable that once you get too many (some people report it happens at something over 500 saves), Morrowind loses its mind. Anyway, people post with this problem from time to time (which you can see if you search for that message), and I thought that if the MCP could fix it, that would be cool.

Thanks.

That problem have also been confirmed by me too. :brokencomputer:

There is another bug that still exist in both previous version of MCP (1.6 & 1.7) and the latest MPP 1.6.5 that is related to lockpicks, probes and all repair tools (all levels) that's for the players equipment even the "Sirollus Saccus Hammer" is affected.

You may ask how this bug is activated by the player and the reason for that it's triggered when you sell any of these misc items to a merchant or trader if the item have fewer uses than 5 left and they are restocked by that trading npc.
Please note I've only tested with a fewer uses than 5 mostly so it could even higher uses left, but you must have use that before.

After you have sold it twice that item is alone in the traders goods inventory with the *selling* uses left and then you buy it back you have an untouched item with all the uses left. :icecream:

Another feature that was included in MCP 1.8 is the toggle sneak option. :thumbsup:
If I install the MCP 1.8 must I also have Yacoby's Toggle Sneak mod installed at the same time to use the toggle sneak mode?

Thanks in an advance.

» Mon Nov 22, 2010 11:58 am

leonardo, toggle sneak works on it's own without yacoby's mod.

» Mon Nov 22, 2010 3:59 am

leonardo, toggle sneak works on it's own without yacoby's mod.

That's really great and thanks Enzo Dragon. :clap:

» Mon Nov 22, 2010 7:33 am

One of the worst render engines of the 21st century. That, or has been used by a record number of games with the programmers making the textures. Specular should never have been allowed.

Actually, a correction on that, I've been making stuff in Unreal 3 for a while, and that's probably brought about by how the engine runs. Which is basically to bake as much as it can and use as many instances as possible. I basically svcks on dynamics, but when put together properly looks absolutely awe inspiring on static environments. You can fit millions of polygons and thousands of lights onto a single map if you want without a significant framerate hit, I've done it. Most of the work is done through materials. If they aren't done properly, everything svcks.

It's also has very powerful shader support - programmers basically.

It's easily capable of photorealism, the only issue is that most people who use the engine aren't entirely aiming for that, and thus don't adjust the lighting/shaders/materials appropriately. Default rendering, is of course that slightly overstated version that you're probably thinking of.

Now. On topic. Hrnnchamd, I happen to have a beast of a pc: 12gig ram, x64bit, Core i7 980x, and am willing to help you with bug testing for x64 stuff, especially with the possible options of using multi-threading and more ram when running, as Morrowind is currently using far too little for what it has to do. I'm getting lag, and I really, really, really shouldn't be.

Even Crysis couldn't drop the framerate on this - though I dunno what cryengine 3 would do to it :shocking:

» Sun Nov 21, 2010 11:17 pm

Hrnchamd, I've read that Morrowind does not run well on DirectX 10 and there is a recommendation in the sticky in the hardware section to run on 9c to avoid CTD - have you found any evidence to support this?

» Sun Nov 21, 2010 9:19 pm

It's quite unlikely there is a difference, unless there are driver bugs. I wouldn't assume that recommendation is universally applicable.

» Mon Nov 22, 2010 11:36 am

Has the problem with dialog variables ever been discussed? Specifically, the ^Name variable that is supposed to be the name of the calling actor but instead defaults to the name of the PC everywhere but in dialog. It would be nice to be able to use it to identify a speaker when using MessageBox.

Hrnchamd, thanks for the all the work you've done on the MCP.

» Mon Nov 22, 2010 10:22 am

For some reason, the restore attributes fix isn't working in my game, despite having checked it when I ran the installer. Has anyone else had this problem?