[WIPz] VoodooSigil

Post » Wed May 09, 2012 3:15 am

tl;dr: The Voodoo Shader Framework is a graphics extender that works with multiple games, allowing the same shaders to run on all of them. It's capable of post-processing (fullscreen, things like bloom and SSAO) and materials (shaders attached to a texture). The difference from other shader mods is plugins (Voodoo uses them to support games and can load extra ones, like script extenders), wider support (not just one game, same shaders work in any game with a plugin) and faster (for simple stuff, most testers report no FPS hit). Fast, simple, free, open-source, well-documented and incredibly flexible. Wiki, code, and other info at:

http://www.voodooshader.com
Comprehensive graphics framework for adding and extending shaders during development or after release.


If you're interested in more details about Voodoo, please keep reading (or skip to the bottom for links). Anyone with suggestions is encouraged to post. Anyone willing to help test, artists interested in what Voodoo can do for mods, or programmers curious about contributing code, please check the bottom of the post.

Intro:
After working a bit on the Morrowind Graphics Extender, porting a few shaders to the Oblivion one, and then writing an extended equivalent for Neverwinter Nights, I got tired of the constant porting and tweaking and incompatible bits. So, after a few requests for graphics extenders for other games (the basics are quite boring), I decided to put together a system capable of handling almost any game. After much trial and some error and many suggestions, the current system was born. The framework has been under development for some time, with a wide variety of outside influence keeping it sane and stable (primarily from friends and coworkers who have experience with similar projects), and the core elements are complete and tested.


What is the Voodoo Shader Framework?
The Voodoo Shader Framework is a large chunk of code to make graphics mods easier.

Instead of redundant and conflicting systems for each game, it uses a single universal core and plugins to fine-tune support for each game. The core handles loading config files and shaders, so one shader will work in any game with a Voodoo plugin.

In addition, a lot of maintenance and upkeep tasks are handled through Voodoo. Tools to help work with game resources, images, and even downloading updates to plugins are all provided. None of them are required, but they are there to make life easier.


What can Voodoo do for me?
It can improve the graphics of your old games and help you tweak the new ones.

For games that already have graphics extenders, it can often provide more features or better performance. With the configuration and update tools, Voodoo is a complete system, not just a few DLLs and some text files.

Perhaps more importantly, Voodoo provides consistency. You get the same tools, the same system, and the same effects across different games. You can set up your favorite look, and then just add on a few tweaks that each game needs.

A few shots of the basic functionality across a few games:
https://www.voodooshader.com/screenshots/morrowind-1.jpg
https://www.voodooshader.com/screenshots/oblivion-1.jpg
https://www.voodooshader.com/screenshots/skyrim-1.jpg
https://www.voodooshader.com/screenshots/fallout3-1.jpg
https://www.voodooshader.com/screenshots/nwn2-1.jpg
https://www.voodooshader.com/screenshots/gothic3-1.jpg
Same code, same shader, different plugins and settings.


How does Voodoo work?
A lot like other graphics mods, but less one big blob.

The main chunks are the core and the plugins. The core doesn't vary between games; it provides the basic shader loading, texture handling, and other important basics like logging and plugin loading. The plugins do the detail work, tying the core into the game. Plugins handle drawing your effects at the right time to get them under the menus, and can even provide improved water or other effects. All Voodoo plugins must provide a minimum set of features, to ensure compatibility across the system, but are free to do more for their game.

Each game plugin has a title related to the game name, plot, or something clever someone came up with. For Morrowind, its Gem, Oblivion is Sigil, Fallout 3 is Raven, Neverwinter Nights is Frost, and Neverwinter Nights 2 is Shard. The working title for Skyrim is currently Storm, but this isn't definite yet.

Voodoo provides two methods of loading into games. One is a Fraps-like GUI which is left running and loads into games as they start up. The alternative is a drop-in DLL, which some of the older graphics mods use. Both are provided and tested, so it's a matter of preference.

The Voodoo GUI also provides a simple method to download updates to the framework, or new plugins. This is all handled over a secure connection to the main Voodoo server, by simply downloading lists of available plugins (no personal data is uploaded). For those wanting to do it manually, downloads links will also be made available for new releases.


When/Where can I get Voodoo?
The framework isn't quite done yet, but I am looking for folks able to help test. Previous testing has gone great, and the latest upgrades are almost ready. The Skyrim module is partially complete, sharing a lot of basic code with Oblivion and other DX9 games. Testing will be done semi-closed at first, with a small group verify that things work before the public testing release.

I will be keeping this thread, and the others for other games, up-to-date with progress as it comes. Release announcements will be posted as necessary.



For the time being, anyone interested in any way is encouraged to get in touch. Post, PM, email, chat, whatever. This thread is open for suggestions, comments, and requests. I have a simple https://github.com/peachykeen/VoodooShader/issues, set up for everything, on the https://github.com/peachykeen/VoodooShader. If you just want to drop by an suggest something, Voodoo stuff is often discussed in the #voodoo (and hijacked #morrowind) channels on the Chatspike IRC network. You can also send over a PM or email.

A https://github.com/peachykeen/VoodooShader/wiki/FAQ is located on the project wiki, with hopefully a decent variety of answers. Any unanswered questions, please post or send them in.

If you are an artist interested in what Voodoo can do for your work, please get in touch. There's quite a bit of variety, primarily the material shader system. It is designed to attach shaders to textures, providing effects beyond what the game itself would.

For any programmers curious about Voodoo, you can see the framework documentation here and all source code is available at https://github.com/peachykeen/VoodooShader. Voodoo is completely open-source under the Lesser GPL, providing plugin authors the freedom to use whatever license they feel comfortable with. It is also designed to be widely compatible with other programming languages beyond the native C++.

For those curious as to the history, some simple graphics are automatically generated from the code on https://www.ohloh.net/p/voodooshader. They don't give much detail, but progress has been steady for a while now and growing slowly as I add modules and work on the design.


Again, questions or comments or contributions of any kind, please post or get in touch. I will be following and updating this thread as I can.

Cheers, peachykeen

Edit Jan 7 2011:
After a few ridiculously complicated bugs (which is what I've taken to calling typos) and a New Year's that I don't fully recollect (hopefully everyone else had good holidays), testing can finally commence! :biggrin:

To begin with, I redid the website a bit with more info on what Voodoo Shader is and some shots of it in action: https://www.voodooshader.com

Currently supported games include Morrowind, Oblivion, Skyrim, Fallout 3, Fallout: New Vegas, Neverwinter Nights 2 and Gothic 3. Testing can be done in any of these, but make sure to note which one you were using.

In a joint test of the (already mostly tested) UI/package manager and Voodoo framework, there are three steps to the process. First, verify you have .Net 2.0 and MSVCR10 installed (links provided). Second, download Voodoo itself. Third, run a game. Detailed instructions are provided.

Testing Instructions: https://github.com/peachykeen/VoodooShader/wiki/Voodoo-Test-Instructions

Please follow these as closely as possible. If you have any questions, comments, or really anything, let me know. That can be done by post, PM, email or on IRC (chatspike.net, #voodoo or #elderscrolls, say my name or voodoo to highlight me).

There are a few issues still hanging around, I'm sure. One known bug, which seems to effect only Morrowind and Oblivion, is a crash-on-exit. This is because of the game not shutting down D3D, which causes some confusion when destroying resources. If you run into any others, let me know.

And before anyone asks, the included shader is intended to be obvious for testing purposes; you just can't miss it. :wink:
User avatar
Loane
 
Posts: 3411
Joined: Wed Apr 04, 2007 6:35 am

Post » Wed May 09, 2012 2:49 am

Whoops! I totally forgot about your PM :facepalm: Will get right to it!
User avatar
Nicholas C
 
Posts: 3489
Joined: Tue Aug 07, 2007 8:20 am

Post » Tue May 08, 2012 4:42 pm

Whoops! I totally forgot about your PM :facepalm: Will get right to it!

Heh, no worries, I have the same problem. ;) I'd like to talk to you folks and see if we can work something out; I'd hate to be stepping on toes or recreating the wheel just to get shaders working in more than one game. :)
User avatar
Angelina Mayo
 
Posts: 3427
Joined: Wed Jan 24, 2007 4:58 am

Post » Wed May 09, 2012 3:44 am

Heh, no worries, I have the same problem. ;) I'd like to talk to you folks and see if we can work something out; I'd hate to be stepping on toes or recreating the wheel just to get shaders working in more than one game. :)
PM sent, as promised!
User avatar
Dorian Cozens
 
Posts: 3398
Joined: Sat May 26, 2007 9:47 am

Post » Tue May 08, 2012 3:07 pm

Interesting project, looking forward to seeing what you and hopefully the OBGE team come up with.
User avatar
helen buchan
 
Posts: 3464
Joined: Wed Sep 13, 2006 7:17 am

Post » Wed May 09, 2012 3:19 am

I'd be willing to do some testing for you.
User avatar
Damned_Queen
 
Posts: 3425
Joined: Fri Apr 20, 2007 5:18 pm

Post » Tue May 08, 2012 2:29 pm

So would I.

I've got a heck of a lot of mods working together just fine, fwiw. And have been known to experiment *a lot* with OBGE, and even ENBSeries. Not that I'm a modder at all, mind you. But the interest is there! :)
User avatar
Alex Vincent
 
Posts: 3514
Joined: Thu Jun 28, 2007 9:31 pm

Post » Tue May 08, 2012 7:04 pm

I've been setting up some test stuff and a few PMs have gone back and forth between shadeMe and I. As soon as we can figure out something there, I'll update this thread with how things will be setup. I've also been doing some testing on the various adapters and have gotten the exact same shader working in a few different games, so that part is working great at the moment. Just have to sort things out with OBGE and we can get started with testing. :)
User avatar
Leah
 
Posts: 3358
Joined: Wed Nov 01, 2006 3:11 pm

Post » Wed May 09, 2012 2:10 am

After a few ridiculously complicated bugs (which is what I've taken to calling typos) and a New Year's that I don't fully recollect (hopefully everyone else had good holidays), testing can finally commence! :D

I have or am sending out the info to everyone who volunteered, as well as posting it here.

To begin with, I redid the website a bit with more info on what Voodoo Shader is and some shots of it in action: https://www.voodooshader.com

Currently supported games include Morrowind, Oblivion, Skyrim, Fallout 3, Fallout: New Vegas, Neverwinter Nights 2 and Gothic 3. Testing can be done in any of these, but make sure to note which one you were using.

In a joint test of the (already mostly tested) UI/package manager and Voodoo framework, there are three steps to the process. First, verify you have .Net 2.0 and MSVCR10 installed (links provided). Second, download Voodoo itself. Third, run a game. Detailed instructions are provided.

Testing Instructions: https://github.com/peachykeen/VoodooShader/wiki/Voodoo-Test-Instructions

Please follow these as closely as possible. If you have any questions, comments, or really anything, let me know. That can be done by post, PM, email or on IRC (chatspike.net, #voodoo or #elderscrolls, say my name or voodoo to highlight me).

There are a few issues still hanging around, I'm sure. One known bug, which seems to effect only Morrowind and Oblivion, is a crash-on-exit. This is because of the game not shutting down D3D, which causes some confusion when destroying resources. If you run into any others, let me know.

And before anyone asks, the included shader is intended to be obvious for testing purposes; you just can't miss it. ;)

Thanks to everyone who volunteered, or tests. Hope to hear back soon. :)
User avatar
Jah Allen
 
Posts: 3444
Joined: Wed Jan 24, 2007 2:09 am

Post » Tue May 08, 2012 8:49 pm

Now for this week's news, mostly good:

The previously-untested deferred rendering system turns out to work just fine. Where most graphics extenders only draw to a single preset texture, Voodoo Shader allows effects to create textures and specify which they want to draw to. This allows for a lot of effects that were previously impossible and improvements to some simple effects, such as bloom and the rest of the blur family. :wink: For those partial to diagrams and charts:

http://i.imgur.com/fc5Zw.png
http://i.imgur.com/ARVdB.jpg

With multiple passes, techniques, and effects all accounted for and NLPP done, we're quickly coming up on the point where this stops being an experiment and turns into a usable project. :smile:

At the request of a few folks, I will be doing what I can to make Voodoo Shader compatible with Wine and Crossover, and if possible will use the framework to help patch graphics and make games run better on those platforms. That's going to take a bit of setup work, but initial checks suggest things aren't entirely broken on Wine as is.

I declared yesterday muffin and benchmarking day. The muffins were delicious and the benchmarks promising. Running a small scattering of games with some reasonably high settings intended to make Voodoo Shader choke up turned out to make it look pretty good.

Frames per second: avg (% difference), min-max. https://docs.google.com/spreadsheet/oimg?key=0AtluffkFk8FedEFHeF9FR2RUa24za2kyYUdVUnJzM1E&oid=5&zx=vmhdjbz344pa, https://docs.google.com/spreadsheet/ccc?key=0AtluffkFk8FedEFHeF9FR2RUa24za2kyYUdVUnJzM1E
  • Skyrim, none: 48.48, 38-61
  • Skyrim, VS copy: 47.42 (-2.2%), 37-60
  • Skyrim, VS bloom: 45.55 (-2.37%), 37-61
  • Oblivion, none: 90.77, 82-95
  • Oblivion, VS copy: 88.10 (-2.94%), 80-93
  • Oblivion, VS bloom: 88.62 (-2.37%), 86-90
  • Morrowind, none: 70.12, 48-109
  • Morrowind, VS copy: 98.37 (40.29%), 69-144
  • Morrowind, VS bloom: 95.68 (36.46%), 76-133
  • Fallout 3, none: 59.55, 55-61
  • Fallout 3, VS copy: 59.37 (-0.31%), 57-61
  • Fallout 3, VS bloom: 58.95 (-1.01%), 57-60
  • NWN2, none: 40.08, 40-41
  • NWN2, VS copy: 38.10 (-4.95%), 38-39
  • NWN2, VS bloom: 35.80 (-10.69%), 35-36
Results are about what you'd expect, more works is less speed. NWN2 by far takes the biggest hit, with Fallout 3 hardly flinching. The Morrowind results are by far the most interesting. I ran them twice, using a 120-second interval the second go, to verify the numbers and they checked out. I suspect the upgrade from D3D8 to 9 is involved in part, but I did not expect the findings. Whether this is good science or I screwed up somewhere, I'm not sure.

In all cases the games were running the highest default settings but no AA or AF (no INI tweaks, ultra or ultra high in the launcher, or maximum from the options menu); all but Morrowind ran in 1920x1080 fullscreen, Morrowind was 1280x768 windowed. Each benchmark ran for 60 seconds, recording FPS for each second using Fraps, and the resulting samples were run through some statistical anolysis in Excel. For Skyrim, the 60 seconds started when the word "presents" appeared in the intro of a new game, and no user input was given. Oblivion was recorded outside of the sewer exit. Fallout 3 was recorded looking out over the wasteland outside of the vault. NWN2 was recorded in the town of Ember, with the camera zoomed as far out as possible and at a low angle. Morrowind was recorded standing on the wall of Balmora, across the river from the silt strider and looking into the city. Each benchmark was run once without Voodoo Shader, once with a basic copy effect (using a shader to show the screen, no special effects), and once with a bloom effect using 6 passes (1 filter, 4 blur, 1 combine). I'll be adding MGE/OBGE and ENB benchmarks when I have time.

Unfortunately, after not a little time and code put into it, I've had to remove Cg from Voodoo Shader. Testing brought to light the sad fact that when it doesn't like something, it crashes, and this behavior is intentional. In this case, that means the whole game crashes, which is simply unacceptable. The final nail in the coffin was when Cg prevented me from even starting OpenGL games with Voodoo Shader, the exact thing it was supposed to make easier. About half of this past week has been me stripping it out, and that effort is now complete and the framework is working again and has a few more features. It would have been nice to use Cg, but I'm not going to even consider something designed to crash your computers.

In place, I'm using DirectX's own compiler and the current plan is to use MojoShader for OpenGL support. This has the side benefit of allowing MGE (XE) and OBGE shaders to be used with almost no modifications at all. For end users, there will be little or no difference.


There will be a new version with some added features within the next few days, probably, so check for updates if you feel like playing with things, and that's about all for now.
User avatar
Melly Angelic
 
Posts: 3461
Joined: Wed Aug 15, 2007 7:58 am


Return to IV - Oblivion

cron