Two things first: The images are quite large so please be patient while they load (they are this large so that you can see the details) and I run the german version of photoshop. But it should be obvious what I mean.
Quick'n Dirty Tutorial: Creating Normal Maps using the Layer TechniqueIn many cases I recommend creating a height map before creating the actual normal map. This is a greyscale image which contains height information. In some cases it's enough to turn you image into greyscale, but you'll get better results when you edit it by hand afterwards. Let's do an easy example: Take your image, turn it into greyscale (quick'n dirty: Crtl+Shift+U), create as many empty layers as you need to add height info and then paint them with black, grey and white, set an appropriate blend mode (soft light, overlay... experiment with them). At the end you'll have something like this:
http://i247.photobucket.com/albums/gg135/Bomret/Heightmap.jpg
Notice: There are people who favour using the dodge and burn tool. No problem with that.
Now some explanation: The grey stones will be "normal level", the brighter ones will stick out more the darker ones less. Easy to understand, I hope. That'll look much more lifelike ingame, because in such an irregular brickwall not all stones are the same. But that's important for other types of textures too. See http://www.gamesas.com/bgsforums/index.php?s=&showtopic=799551&view=findpost&p=11625784 to get an idea.
Now on to the normal map itself. The idea is to duplicate the original layer a few times, then put on a gaussian blur on each layer with an increasing amount, except the top one. After that run the normal map filter on each layer with an increasing scale from top to bottom. Sounds complicated? I hope I'm able to clear the confusion with some screens. First, it's necessary to put the image into a matrix of itself. Just enlarge the canvas size and layout the image next to each other. That's important to avoid evil gaps in your normal map. Here's what I mean:
http://i247.photobucket.com/albums/gg135/Bomret/HeightmapMatrix.jpg
The height map is in the center, surrounded by itself. When the normal map is finished, we'll reduce it to it's original size.
Time for step-by-step:
1. Put the heightmap into a matrix, afterwards flatten the image.
2. Duplicate the image five times. At the end we'll have six layers showing the same http://i247.photobucket.com/albums/gg135/Bomret/Ebenentechnik.jpg.
3. To be able to see what you're doing next, hide the top 5 layers with a click on the eye next to them so that only the bottom layer is visible. Now comes the Gaussian Blur. Activate the bottom layer and run "filter->blur->Gaussian Blur" and use a value of 10. Then unhide the next layer and repeat the Gaussian blur. But this time use a value of 8. Next layer gets a 6 and so on. Reduce the value of the Gaussian Blur by 2 on every layer. So from bottom to top that would be 10->8->6->4->2. The top layer is left unchanged.
4.Now hide all layers except the bottom one again. It's time for the normal map filter. Activate the bottom layer and choose "filter->Nvidia Tools->Normal map Filter". In this window set up the following:
- Filter Type: 4x4 Sample
- Wrap
- Scale: 80
- Height Source: Average RGB
- Alpha Field: Set to 1.0
Click OK and we'll get the following http://i247.photobucket.com/albums/gg135/Bomret/Ebenentechnik2.jpg.
As you can see there are some errors like small "stairs". That's normal for an 8bit normal map. You can avoid that by working in 16bit. But on the one hand the whole thing would take longer and on the other hand we'll lose color information when converting it back to 8bit, which may lead to ugly errors. Instead we just use the Gaussian Blur again with a value that eliminates the small stairs (in most cases a value from 1 to 4 is enough. I used 2 in this example). The procedure is the same for the next steps:
5. Unhide the next layer and run the normal map plugin. This time use a scale of 60. If errors occur, jsut blur them away with the Gaussian Blur. Now put the blend mode to "hard light". With doing that our layer is added to the bottom one and we gain more detail with every layer.
6. Do this with the remaining layers. The only thing that changes is the scale of the normal map filter. In our example from bottom to top: 80->60->40->20->10->6. Every layer gets the blend mode "hard light", if the "small stairs" appear, blur them away.
http://i247.photobucket.com/albums/gg135/Bomret/Ebenentechnik3.jpg
7. Flatten the image. Now we can reduce the image back to it's original size. Just use "image->canvas size". It's important that the center square is http://i247.photobucket.com/albums/gg135/Bomret/Ebenentechnik4.jpg.
The last step is Re-Normalization. Until now we treated our normal map like an image. But that's not the case. For the game engine it's a map that contains mathematical information, not colors. Re-Normalization is simple: Open up the normal map Filter and click on ?Normalise Only". Your normal map may lose some brightness, but that's ok. You won't lose anything. Now our blue channel also gets it's height information. Now we can save it in .dds format.
That's the wall ingame (I created the color map earlier):
http://i247.photobucket.com/albums/gg135/Bomret/Screen1.jpg
If you put more time into that, you'll get much better results, that's just a quick'n dirty example. I love the layer technique because it's extremely flexible. It's up to you how many layers you use and which values for the Gaussian Blur and Normal Map Filter. You can do any normal map with it and get astonishing results. Just experiment
A last tip: For the last two layers (the two top ones) do not use the height map but the original image as a source. In many cases you preserve more detail.
» Sat May 28, 2011 10:25 am 
I've seen/used 4096 textures and have them render just fine, it never occoured to me that it may never use the 'top' level image. I guess for testing, make an all white image, save it with mipmaps. Then re-open it, change the base image to black, and save with the option of "use existing" mipmaps. If you never see the black image, then you'd know if Oblivion discards over a certain limit. *shrug* (and in case you're curious, I do believe saving 64bpp images worked, but I do remember that 128bpp ones didn't.)
