[WIPz] Better Colors

Post » Mon Jul 05, 2010 10:40 pm

Yet another of these from me. 'cept this one is useful. :P

All the gallery screenshots from the Bitter Coast and the Ascadian Isles that get so many oohs and aahs have one thing in common: palettes (usually).
They use custom tinting and color-shifts to give the area atmosphere and boost the environment.

This works great, except it makes Balmora turn bright green too.
So, with a little programming work, I've whipped up a simple solution to this. Using MWSE, MGE and their script functions, fully dynamic cell-based tinting is now available.
This can be used to add special tint or new fog colors to your horror mod, swamp, or just make the Bitter Coast always green and Solstheim always icy blue.

The system consists of a small mod with 2 global scripts, a single simple shader, a small program and a ruleset.
From my testing, there was absolutely no FPS hit (on some systems running the shader might cost 1 or 2 FPS). The ruleset is loaded by the program, which then communicates with Morrowind through a pipe, and the scripts pass the info on to the shader. Because the lookups (which are the longest part of the process, sometimes taking as much as 0.025 seconds) are done only when the cell changes and are performed asynchronously with Morrowind, the scripts are designed to request new color info, let the game run for a moment, then process the data. This leads to very few script commands being executed in any given frame, making the scripts run pretty darn quick.

This system is exceptionally mod-able and flexible. The current cell and player's position are sent to the program each time the cell changes, and it uses some internal methods to find the best color to send back. The rules are human-readable and come from a simple text file, which has paired match strings and color rules.
Three types of matches are supported:
- Simple matches (if the match is "Seyda Neen", then any cell with that exact name will match)
- Cube matches (uses the player's position to match, if they are within a defined volume)
- Regex matches (besides having to use regex, since it's the solution to everything, this allows for wildcard and patter matches)

For example, if you wanted to match any wilderness cell (there are a lot), the match would be:
- Wilderness

- indicates a simple match, then "Wilderness" (case-sensitive) is the name. For something more complex, you could use the regex match:
@ Vivec(.*)

to match any cell that starts with Vivec (this will match the whole town area, essentially). For something more detailed, you could use:
^ -1000, -1000, 100, 1000, 1000, 1000

to match any player that is within a 2000-unit square around origin, between 100 to 1000 units above water-level.

I'm working on making things even more flexible (perhaps an update interval) and will be adding combined cube/simple and cube/regex rules here soon (filter for areas within a cell).
There's no transition between colors at the moment and the shader is still set up for some debugging, so you'll get a colored band across the left edge of the screen, but otherwise, this seems to work. I just need testers to try it out for me. :)

Edit: The download may not be working, though it looked like it when I was playing. If the shader doesn't change color, let me know. I think the debug mode of the program may be broken at the moment.
http://www.mediafire.com/?oganyzydnqk

Install Instructions:
- Download
- Extract
- Move the ESP and shader folder to your Data Files directory
- Enable the ESP

Playing Instructions:
- Start MWFogProject.exe
- Start Morrowind
- Play

If the scripts work, you should see the shader load and some occasional debugging messages. I'll be removing those before release, obviously.
If the program and everything else works, you'll see the world and band change color when the cell changes.
There are only a few rules in the db at the moment, for Seyda Neen, Vivec, and a few altitude levels (the world will turn more blue as you go up).
I haven't bound fog properly, so at the moment the fog color isn't changing, but the tint will and is usually noticeable.
Once it's done on the tech side, I'll be putting together a full db of colors for the stock game and expansions, as well as any mods I have.

Any testing will be greatly appreciated.
User avatar
Elena Alina
 
Posts: 3415
Joined: Sun Apr 01, 2007 7:24 am

Post » Tue Jul 06, 2010 12:37 am

Nice idea :)

Seems to work great.
User avatar
Alba Casas
 
Posts: 3478
Joined: Tue Dec 12, 2006 2:31 pm

Post » Tue Jul 06, 2010 3:50 am

So if this is tied to areas of the map, mod added land (looking at you, TR) will be normal. This ought be fixed... by someone other than myself, scripting is not my forte.
User avatar
Mr. Allen
 
Posts: 3327
Joined: Fri Oct 05, 2007 8:36 am

Post » Tue Jul 06, 2010 3:40 am

The .exe isn't working for me (it closes automatically) so everything looks messed up. Could it be because I'm using the steam version, so the path is changed?
User avatar
Brooke Turner
 
Posts: 3319
Joined: Wed Nov 01, 2006 11:13 am

Post » Tue Jul 06, 2010 3:43 am

Updated it:
http://www.mediafire.com/?yyygjag2cmm
It now has more rules, more types of rules (you can combine and search for an area within a cell), and a configurable interval to test at.
Use
Set pk_fog_interval To #secondbetweentests

to set that up.
The only bug I've seen is when it takes the client too long to find the rule and return it. I've only had this happen with an infinite loop I accidentally coded in, though, so far it's never failed to return the results within the timeout (it's 0.1 seconds at the moment). That's actually mildly impressive, seeing as it now tests for a couple regular expressions and a few cube areas.

Nice idea :)

Seems to work great.

Good to hear. :icecream:


So if this is tied to areas of the map, mod added land (looking at you, TR) will be normal. This ought be fixed... by someone other than myself, scripting is not my forte.

This is true. If it used traditional scripting.
Instead, it's set up so the little program uses a rule database. So you can add rules for the TR cells (which I'll definitely do before I release a final version) and they just won't be called unless the player's in the cell. I'm going to add rules for most major areas (Morrowind, Tribunal, Bloodmoon, TR, Silgrad, etc) and it won't matter if you have the mod or not, no crashes if you don't. This has no mod dependencies at all, it basically runs parallel to all mods and can work with any that are present.


The .exe isn't working for me (it closes automatically) so everything looks messed up. Could it be because I'm using the steam version, so the path is changed?

That shouldn't matter, as it uses a named pipe instead. The exe shouldn't be closing though, unless something is messed up. Do you see any error messages? If you run it in a command prompt (start, run, type cmd, hit enter, find the program and run it), what do you see?
The program should start,give you a quick message and sit there until MW starts. After the script hooks up, it'll start spitting out more useful data and send stuff on.
User avatar
emma sweeney
 
Posts: 3396
Joined: Fri Sep 22, 2006 7:02 pm

Post » Mon Jul 05, 2010 11:19 pm

I'm getting a lot of something 1 errors (don't quite remember the name) it's friggin early in the morning here.

Loving the fact that this idea is finally getting some love!

I have some really wacko screenshots with mono red skies. Also sky seems to dissapear all together. Hmm.

Will try some things further.
User avatar
Albert Wesker
 
Posts: 3499
Joined: Fri May 11, 2007 11:17 pm

Post » Mon Jul 05, 2010 6:08 pm

Will try to test this when I get back in later, but for now, just wanted to say nice job :foodndrink: ! The palette idea has been tossed around, as you said, but this is the sort of thing that's been needed badly to manage them.
User avatar
Luis Reyma
 
Posts: 3361
Joined: Fri Nov 02, 2007 11:10 am

Post » Tue Jul 06, 2010 12:10 am

Any screens?
User avatar
Veronica Martinez
 
Posts: 3498
Joined: Tue Jun 20, 2006 9:43 am

Post » Tue Jul 06, 2010 4:16 am

I've updated it quite a bit, and here's the latest:
http://www.mediafire.com/?znlgjmncnmm

It's almost getting to the actually useful point. The ESP still has debug messages, and so does the app, but it functions well in most cases. I'm mainly testing wide compatibility and speed.
My benchmarks show that under heavy system load (Morrowind running in a window, Fraps just starting recording, MGE with distant land and a couple shaders, iTunes running in the background and Visual Studio debugging the fog program) the worst lookup times were around 30 milliseconds. Times were 15 ms while loading distant land. Usual time is <1 ms on my system. The timeout in the script is set to 0.015 seconds and can be changed if you run into problems (float pk_fog_timeout).

The new ESP has semi-intelligent interval updates (if it encounters an error it will stop updating), on-cell-change updates, and compatibility with a modified version of phal's ENB palette shader (thanks to skaeps.mp3 for some help with that). It also has slightly improved tint shader (Color Mood) support. Both the palette and tint shader can be run together, if desired.
Included are 3 palettes, which the shader will automatically shift between (as if they were points on a triangle). Palettes need HDR to be turned on to work right, so I made them optional and you can disable that shader and updates to it by setting the variable pk_fog_palette to 0 in the console.

Rules have been extended to use an 8th parameters (palette strength). The current rules aren't complete, so you may often get red debugging settings. Underwater and at high altitudes, you'll get varying levels of blue and there are rules for Seyda Neen and Vivec in place.

I'm getting a lot of something 1 errors (don't quite remember the name) it's friggin early in the morning here.

Loving the fact that this idea is finally getting some love!

I have some really wacko screenshots with mono red skies. Also sky seems to dissapear all together. Hmm.

Will try some things further.

Read.1 errors? It means the script failed to read from the program at the first stage (it reads tint values, then palette values, in two separate reads of 4 floats each). Make sure you have the program running. You should start the app before Morrowind starts, make sure it says connecting to pipe in the little console window, then start MW.

I think I fixed the sky issues in this latest, not sure. I turned off manual lighting in favor of pure tint and palette shaders, so no more fog tweaking, but it does allow for Vality-esque palettes. Except, more than one palette and dynamic shifting. Some of the rules are also a little over-the-top for testing purposes, so I'll make them more reasonable and it should look better.

Will try to test this when I get back in later, but for now, just wanted to say nice job :foodndrink: ! The palette idea has been tossed around, as you said, but this is the sort of thing that's been needed badly to manage them.

I've integrated the good ol' ENB palette shader (what Vality's screens used, put in MGE thanks to phal, added to this thanks in part to skaeps.mp3), with a couple custom palettes, so hopefully it'll be even more useful.


Any screens?

Will be forthcoming (with videos) momentarily (converting now). :)
The debug settings are rather ridiculously bright, though, so it's mainly for proof-of-functionality now and I'll try to get some good-looking shots later.
User avatar
rebecca moody
 
Posts: 3430
Joined: Mon Mar 05, 2007 3:01 pm

Post » Tue Jul 06, 2010 1:47 am

snip

Will test the new upload. However I did double check the previous one and all of the criteria mentioned above were met so it should not get that error. Hmm.
User avatar
Hayley Bristow
 
Posts: 3467
Joined: Tue Oct 31, 2006 12:24 am

Post » Mon Jul 05, 2010 8:29 pm

I can't wait to see screens of this.....
User avatar
Riky Carrasco
 
Posts: 3429
Joined: Tue Nov 06, 2007 12:17 am

Post » Tue Jul 06, 2010 12:12 am

Tried to run the updated version and I'm getting this error set:
http://img25.imageshack.us/img25/9448/mgescreenshot2.jpg



I have the .exe running in the background, and it says that its pipe was established. But no luck running in game :( . Am I overlooking something simple?
User avatar
Chloe Botham
 
Posts: 3537
Joined: Wed Aug 30, 2006 12:11 am

Post » Tue Jul 06, 2010 12:01 am

I can't wait to see screens of this.....

Screens:
Neutral colors (default): http://i4.photobucket.com/albums/y145/peachykeen000/bettercolors/MGEScreenshot095.png
Warm green palette/tint: http://i4.photobucket.com/albums/y145/peachykeen000/bettercolors/MGEScreenshot094.png
Mid-altitude red-reduction: http://i4.photobucket.com/albums/y145/peachykeen000/bettercolors/MGEScreenshot098.png
Slight red-reduction: http://i4.photobucket.com/albums/y145/peachykeen000/bettercolors/MGEScreenshot097.png
Blue-tint: http://i4.photobucket.com/albums/y145/peachykeen000/bettercolors/MGEScreenshot096.png
I tried to catch the settings message in all of those. Some of them are very similar rules.

I also have a few videos, though the framerate is terrible (Fraps, especially with my current drivers, kills speed :( ). This one shows off what the mod does pretty well, though:
http://www.youtube.com/watch?v=7SKBDOLjAwk

The package I used to take those shots/video is here:
http://www.mediafire.com/?y01z4w05ncg

If you've ever used the ENB palette shader or Color Mood shader (which I took from some CryTek GDC publications), the effect should look familiar. I've simply made it dynamic. :)

Will test the new upload. However I did double check the previous one and all of the criteria mentioned above were met so it should not get that error. Hmm.

Tried to run the updated version and I'm getting this error set:
http://img25.imageshack.us/img25/9448/mgescreenshot2.jpg

I have the .exe running in the background, and it says that its pipe was established. But no luck running in game :( . Am I overlooking something simple?

I'm not sure what the issue is... Do you both have the internal MWSE running for MGE? That error is definitely related to communication between the script and program, but there are a few things that could be causing it. I haven't had any problems, but it may be related to operating system bit-ness or something unusual. I'll do more testing on my end and see if I can figure anything out.
User avatar
mike
 
Posts: 3432
Joined: Fri Jul 27, 2007 6:51 pm

Post » Tue Jul 06, 2010 7:54 am

Thats simply.........brilliant!
I can just imagine how it can be used.....
User avatar
Sylvia Luciani
 
Posts: 3380
Joined: Sun Feb 11, 2007 2:31 am

Post » Tue Jul 06, 2010 10:46 am

I'm not sure what the issue is... Do you both have the internal MWSE running for MGE? That error is definitely related to communication between the script and program, but there are a few things that could be causing it. I haven't had any problems, but it may be related to operating system bit-ness or something unusual. I'll do more testing on my end and see if I can figure anything out.


When I tried running it, I had MGE rev163 installed for that particular copy of Morrowind, with internal MWSE enabled. I launched the .exe, saw it's message that the pipe was established, and launched Morrowind.

Let me know if you figure it out, I'll be trying some things on my end too =)


Just for reference, some relevant system information:

MGE: rev163
Windows 7 x64 (UAC disabled, Morrowind installed to a custom directory)
NVIDIA GTX 260
User avatar
Nuno Castro
 
Posts: 3414
Joined: Sat Oct 13, 2007 1:40 am

Post » Tue Jul 06, 2010 3:57 am

Ah it needs MWSE? Will you realise version without MWSE?
User avatar
yermom
 
Posts: 3323
Joined: Mon Oct 15, 2007 12:56 pm

Post » Tue Jul 06, 2010 7:46 am

Thats simply.........brilliant!
I can just imagine how it can be used.....

I'm planning on coloring most stock game areas and some other major mods, but that's all I've got so far. If you have any other ideas for its use, do tell. :)


When I tried running it, I had MGE rev163 installed for that particular copy of Morrowind, with internal MWSE enabled. I launched the .exe, saw it's message that the pipe was established, and launched Morrowind.

Let me know if you figure it out, I'll be trying some things on my end too =)


Just for reference, some relevant system information:

MGE: rev163
Windows 7 x64 (UAC disabled, Morrowind installed to a custom directory)
NVIDIA GTX 260

When you first start the EXE, is it immediately saying the pipe is established? On my system it stops at connecting until the script calls it the first time, then it starts showing the updates. Not being able to see what you see, I'm wondering what shows up in the console window. Are any rules visible?
The only major different (that should matter here) is that you're running 7x64. I didn't get a chance to test on that yesterday but I will later today. It's possible the OS is doing something to it. I've used it with MGE rev118 and 161, so I don't think that's breaking it.


Ah it needs MWSE? Will you realise version without MWSE?

No. It's just not possible to do without MGE, and I can't access MGE through scripts without MWSE. In addition, scripting all the rules would be much less flexible, slower and generally not possible (the program I wrote to handle rules does a lot of things MWSE isn't even capable of, much less MWScript).
User avatar
Maria Leon
 
Posts: 3413
Joined: Tue Aug 14, 2007 12:39 am

Post » Tue Jul 06, 2010 12:53 am

the color in the video changes very abrupt. do you intend to make a more smooth transition? but apart of this, your mod is incredibly awesome!

one more question: what is a regex match? is this similar to MWs check for cellnames, if I check for "Mournhold" I actually check for all cells, which begin with "Mournhold"?
User avatar
suzan
 
Posts: 3329
Joined: Mon Jul 17, 2006 5:32 pm

Post » Tue Jul 06, 2010 3:36 am

@NMZmaster, do you have the MwFogProject.exe in the Morrowind folder? Mine had some problems when it was sitting in Data Files, so I moved it. Try moving it and see if it helps.



# DON'T MATCH THE WILDERNESS! Most un-named exterior cells are "wilderness",

I noticed this when I was toying with the script. For my palette script, I was using GetPCCell==X matches, and those will return the region name.
		if ( GetPCCell "Bitter Coast Region" == 1 )			set region1 to 1		elseif ( GetPCCell "Ascadian Isles Region" == 1 )			set region2 to 1                ...                ifx ( region1 )                        setx cellname to xStringBuild "Bitter Coast Region"                ...

Using whatever variable names you use, of course.
It's clunky, but that might help with Wilderness problems.
User avatar
Carlos Vazquez
 
Posts: 3407
Joined: Sat Aug 25, 2007 10:19 am

Post » Tue Jul 06, 2010 11:14 am

Might the tint, when applied, lower FPS, or is it just like those MGE Shaders that merely change color (like Night Vision) that cause basically no performance hit?
User avatar
Tamika Jett
 
Posts: 3301
Joined: Wed Jun 06, 2007 3:44 am

Post » Tue Jul 06, 2010 10:44 am

According to Peachykeen, there is no FPS hit, so you should be good.
User avatar
Emma Parkinson
 
Posts: 3401
Joined: Wed Jul 26, 2006 5:53 pm

Post » Mon Jul 05, 2010 9:30 pm

the color in the video changes very abrupt. do you intend to make a more smooth transition? but apart of this, your mod is incredibly awesome!

one more question: what is a regex match? is this similar to MWs check for cellnames, if I check for "Mournhold" I actually check for all cells, which begin with "Mournhold"?

I'm going to be making it smooth as soon as I get it stable. Toying with a few ways to smooth out the change, but it's going to take some reworking of the scripts.
As for what is regex, there are plenty of pages explaining it, but basically it's a way to match words and patterns of words. A quick example:
The character "." matches any character (letter, number, space, whatever, it represents 1 random character). The modifier * matches 0 or more of whatever came before it. Therefore, the group (.*) matches 0 or more of any character. The modifier ^ matches the start of a string (cell name, in this case). So
^Vivec(.*)
matches any cells that starts with the word "Vivec" and has 0 or more other characters after it. That's the whole city. Regex is a very complicated thing if you use all the features, but it's basically a way to match patterns.
I'm using http://www.boost.org/doc/libs/1_43_0/libs/regex/doc/html/index.html and they have http://www.boost.org/doc/libs/1_43_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html.


@NMZmaster, do you have the MwFogProject.exe in the Morrowind folder? Mine had some problems when it was sitting in Data Files, so I moved it. Try moving it and see if it helps.

In theory, that shouldn't do anything, but it's worth a shot. I generally run the app from a flash drive (M:\mwfog\MwFogProject.exe) and it works fine for me. I'm suspecting more 7-funniness.


I noticed this when I was toying with the script. For my palette script, I was using GetPCCell==X matches, and those will return the region name.
		if ( GetPCCell "Bitter Coast Region" == 1 )			set region1 to 1		elseif ( GetPCCell "Ascadian Isles Region" == 1 )			set region2 to 1                ...                ifx ( region1 )                        setx cellname to xStringBuild "Bitter Coast Region"                ...

Using whatever variable names you use, of course.
It's clunky, but that might help with Wilderness problems.

Hm, thanks. That's a really good idea, I wasn't sure how to get that info out. :foodndrink: I'm not sure how easily it'll be to implement or how fast it'll run, but it may be better than using a million cube matches. Although cubes are still useful, this would make it easier. The only problem is that won't be mod-compatible, where as rules are completely backward and forward and sideways compatible.

Might the tint, when applied, lower FPS, or is it just like those MGE Shaders that merely change color (like Night Vision) that cause basically no performance hit?

Tint should have almost no FPS hit, it's a very simple shader (compiles down to something like 4 commands on your video card). Palettes, however, will probably take off a few frames, that's a more complicated shader and requires HDR, but I made it optional because of all that. There can also be a slight delay when MGE functions are called, I've seen that occasionally in a few mods, but that's hardly noticeable and only occurs every couple seconds.
User avatar
Leanne Molloy
 
Posts: 3342
Joined: Sat Sep 02, 2006 1:09 am

Post » Tue Jul 06, 2010 8:25 am

This looks very promising. What's going on with this Peachykeen? Do you need more testing done?
User avatar
Nicholas
 
Posts: 3454
Joined: Wed Jul 04, 2007 12:05 am

Post » Tue Jul 06, 2010 5:30 am

Any testing on the basic system is helpful. I think the basic rules program is mostly stable. I just have to figure out and write up all the rules for areas.
User avatar
Cagla Cali
 
Posts: 3431
Joined: Tue Apr 10, 2007 8:36 am


Return to III - Morrowind