I've noticed something about some of the texutre mods uploaded to Nexus. This post is to draw attention to good texture mod packaging practices, and generate discussion on techniques artists can use to create such mods.

Disclaimer: I'm not actually a texture artist myself, though I have some limited experience manipulating textures (re: MMM), so those more knowledgeable than me please feel free to chime in!


The Problem
```````````````

There are so many texture mods of all makes and models that have been released so far, some them fantastic and very professionally made. However I noticed, delving into the files, that some of them have textures saved in the wrong format, while others often forget mipmaps. And given Skyrim is already more demanding memory wise than its predecessors, it's more important than ever to keep an eye on memory usage when using texture mods. Unfrotunately texture mods, especially large ones, if saved using the wrong format can waste a lot of valuable video memory real estate.

When video memory reaches its limit, textures are streamed from main memory causing a huge performance loss. There may be users playing now with texture mods that don't realise the sudden FPS jumps and stuttering they're experiencing are due to taxed video memory on their cards.

But it's not just performance that takes a hit. Visual fiedilty can be sacrificed in order to stay within the limits of video memory. Users are told to run at a lower resolution, or turn down their AA, to recover performance -- but if the problem is a poorly packaged texture mod pushing you past your card's limit, then there may not be a need to do this at all. And if you're an author, keep in mind that if your mod is too demanding people may elect not to use it -- and if you made it to be shared and used, you're losing out from having your work bringing joy to the denizens of Skyrim (read: everyone!)


The Example
```````````````

How important is the correct format? By way of example I'm running a good 30 odd texture mods -- some are large replacers (landscape, NPCs, clothing, armor, weapons for example) while others focus on particular areas (objects, architecture etc). I've selectively chosen them to keep within my video card's memory limit -- 1536MB (Nvidia GTX 580).

Now there are a lot of amazing texture mods on Nexus, but a fair few I've seen so far don't always use the correct texture formats. A high-profile and popular mod (and I don't mean to single you out Chris!) that's done this is Chris2102's Whitrerun HQ Texture Pack.

By default this weighs in at 226M zipped, 487M uncompressed. However for whatever reason, while Chris has saved his textures containing an alpha layer and normalmaps correctly as DXT5, there's a large number of textures without alpha also saved as DXT5 when they can be saved as DXT1 for a significant memory saving. Re-compressing Chris's mod using DTX1 where needed gives us:

• Default -- 487M
• Recompressed -- 362M

A saving of 125M -- and remember no image quality is lost. DXT5 and DXT1 look the same, DXT1 just doesn't store alpha. If we presume the majority of these textures are loaded while walking around Whiterun, and if you're already at your video card's memory limit, not only would this prevent stuttering caused by taxed video memory, but it would also leave enough video memory to go from using no AA at all to 8xAA, or enabling SSAO, or using a mod like ENB. Or simply having more video memory to be able to use more texture mods!

And that's just with one mod.

Two other examples I've come across inlcude HQ Towns and Villages where recompressing DXT5 textures without alpha to DXT1 netted a 17M reduction in size (which, given all these textures are used at once, is a saving of 17M in-game) and the Blunt Weapon Enhancement Pack which goes from 90M to a tiny 12M, saving 78M -- here, you might not have all the blunt weapons loaded at once but each one gets a saving of around 3M, so if four different blunt weapons are loaded in a cell that's 12M saved. And, of course, the savings from each of mod add up.

Now before you get too excited, some of the major overhauls -- including the gigantic Skyrim HD and Vurt's Skyrim Flora are both compressed correctly, and there's no savings to be made. But I have, at least in my list, about 18 texture mods -- of just those that I've chosen to download -- that I found I could recompress to gain valuable video memory back.

As an aside, these are all factors that affect video memory usage in addition to textures being loaded:

• Resolution
• Anti-aliasing
• Transparency anti-aliasing
• Triple buffering
• SSAO
• Skyrim: uGrids and extended range tweaks
• Post-processing mods like ENB

Look at this list as a trade off to what you can and can't have if your video memory is approaching its limit -- let alone the use of more texture mods as well. Thus, there's tremendous benefit for texture mod authors to correclty compress and package their mods -- it not only allow users to use your mod in the first place, but as an author the smaller file sizes means it's quicker to upload too. Win!


The Solution
```````````````

So if you're a texture modder, what are the guidelines to follow?

• Remember to generate mipmaps (these help reduce GPU load)
• Save textures as DXT1
• Save textures with alphas as DXT3 or DXT5
• Save normalmaps as DXT5

And

• If you've uploaded a texture mod already, and you're not sure about the formats you've used, please check it and re-upload the mod if necessary.

Deathb0rn has conviniently linked a table of common texture volumes with the resulting file size after compression at various DXT levels -- this way you can usually tell what compression a texture has by its size. http://i255.photobucket.com/albums/hh122/deathb0rnwc3/Skyrim/ResolutionsandCompression.jpg The only thing to add to this is that when a texture is lacking mipmaps, it's usually 33% smaller than the sizes you see in the table.

Again my forays into texture creation are limited, so any input from the experts is welcome here -- for example what about bump maps? Specular? Can normal maps be compressed as DXT1 if they have no alpha? Please share your expertise!

Incidentally, I've been using AMD's The Compressonator to examine textures, generate mipmaps, compare textures and of course recompress them. It's a brilliantly easy tool to use, and its image comparison feature esepcially is fantastic (you can see at a glance what impact different compression levels will have). While I can't confirm this, from what I've read it apparently also produces better quality compressed textures than Nvidia's DDS tools, but I'll leave that for the experts to debate.


The Solution Ver 2 (kinda)
``````````````````````````````
Ethatron has released his DDSOpt tool which does a great job at picking up and fixing (by re-compressing a texture or a normal map correctly) some of the issues discussed above. It also has an excellent mipmap generator that adds higher levels of details in the mipmaps, leading to even better texture quality at mid to long ranges.

So -- for users users and especially texture authors -- you can run DDSOpt to optimise textures to be correctly saved in the right format, and thereby reduce VRAM usage. It is still in development however, so if encounter any odd results please post in the http://www.gamesas.com/topic/1330264-ddsopt-of-use/.


-----> However, there is a large caveat <-----

DDSopt can correctly re-compress textures and can sometimes pick up minor issues such as unnecessary alpha channels, but it is not a silver bullet to magically make all your textures perfect. It cannot fix errors in textures and alpha channels created (however inadventently) by the texture author. Things like left-over borders, stray pixels in alpha channels and a whole bunch of stuff that actual texture artists could probably mention that I can't.

In other words, if you make and upload texture mods DDSOpt can help you ensure your textures are properly saved with the most optimal DXT compression format, but it cannot fix problems with textures, alpha channels and normal map generation. This is up to you to get right first time before you upload the mod -- and unfortunately, while testing DDSOpt with Ethatron, there are a lot of mods that have these sorts of issues and as a result DDSOpt can't optimise them. So if you're a texture author, please check your work

DDSOpt can be http://www.skyrimnexus.com/downloads/file.php?id=5755.

Thankyou for reading.


isoku chimes in:

MadCat221 adds:

teanandcigarettes says:

Ethatron adds:

Huleed gives the straight dope:

Resources
````````````
The Compressonator -- http://developer.amd.com/tools/compressonator/pages/default.aspx
Oblivion DDS guide -- http://cs.elderscrolls.com/index.php/DDS_Files
Wikipedia on DXT modes -- http://en.wikipedia.org/wiki/S3TC

Generating normal maps (thanks to teaandcigarettes) -- http://wiki.polycount.com/NormalMap
More on DXT formats (thanks to teaandcigarettes) -- http://wiki.polycount.com/DXT
Another good read on DXT formats (thanks to Phitt) -- http://www.fsdeveloper.com/wiki/index.php?title=DXT_compression_explained

How to choose the right DXT compression (thanks to Thottlekitty) -- http://cs.elderscrolls.com/index.php/Choose_the_right_DXTC_compression_algorithm
Normal map workflow (thanks to Thottlekitty) -- http://www.poopinmymouth.com/tutorial/normal_workflow.htm
Combining normalmaps (thanks to Thottlekitty) -- http://www.rodgreen.com/?p=4
Throttlekitty's Keep Details profiles for Photoshop -- http://www.tesnexus.com/downloads/file.php?id=11817

DDSopt by Exatron -- http://www.gamesas.com/topic/1330264-ddsopt-of-use/

Couldn't agree more. I go through all my textures I download and re-save them properly. Thanks for bringing this to the attention of, well, everyone

Have you seen any issues with Skyrim HD? I wouldn't know the if there was. (I'm bullocks at anything graphical. )

maybe the ppl dont know if they need to store an alpha or not?

You make some excellent points and thanks for bringing this to everyone's attention. Very well written.

Addenum
```````````

I did actually run this test but forgot to include it above:

Running around Whiterun with Chris2102's Whitrerun HQ Texture Pack

Total video memory load for Skyrim as measured by Nvidia Inspector, at 1920x1200, on my system (with various tweaks, mods, etc):

• Chris' Original pack -- peak 1464M
• Recompressed pack -- peak 1385M

A difference of 79M, which isn't bad for a single mod repackaged mod. That's also the difference for me being able to run SSAO or not (which needs over 60M with ENB's implementation).


No, Skyrim HD is fine (well, there were a few normalmaps compressed as DXT1 instead of DXT5, but they don't have alphas).

If a texture doesn't have an alpha, it doesn't need DXT3 or DXT5 compression. As to creating textures with alphas, that's perhaps best asked of the texture artists here!

I've come across a lot of texture mods that go far beyond the necessary file size and as Omnomnom25 I resave them properly.. I think it would be obvious that something isn't right when your texture is 4times larger than vanilla though its the same resolution. I hope that this thread will be of help to a lot of people who are retexturing stuff.

Small Edit: Maybe this thread could include a filesize reference for the diferent compression methods and resolutions. Don't know if it could be of help, but why not?

Good to know that Skyrim HD is as optimized as it can be. It's still huge though.


That's what she said.

i dont know much about editing textures, so i also would not know if the textures that i have downloaded are compressed well. what about face/body textures or specular maps? if i open a downloaded face texture in gimp, i can remove the alpha channel. so, is this right or wrong? should it be saved as dtx3/5 or as dtx1, because a face texture dont need an alpha channel? or does it actually need one?

What about Xenius Character Enhancement and Realistic Water Textures? Are they optimized like Skyrim HD?

Especially Xenius, expanded the file size is 375MB.

That's an excellent idea. It's nigh on 4am here so I'm off to bed soon but if someone draws it up I'll add it to the OP.


Passing this one off to any texture artists (I'd say if a texture has an alpha channel, it needs it, but I agree I'm not sure why face textures would need one and they have _msn.dds for normal/specular etc).


Also, If it helps and should the authors read this post, these are the other mods (keep in mind -- only of those I've downloaded and use) where the texture formats can be corrected to save memory:

• Bellyaches choice - 14M
• TreesHD Skyrim Variation - 6M
• HQ Towns and Villages - 17M
• Blunt Weapon Enhancement Pack - 78M
• 2K Alchemy and Enchanting tables - 8M
• Tobes High Res Textures - 18M
• Skyrim Redesigned -- ** (depends on what parts you choose)
• HD Furniture and Barrels - 5M (also was lacking mipmaps)
• HQ Food Ingredients - 23M
• Ingots Pack - 9M
• Better Beast Races - 66M (an exception, see note*)

I've also indentified SLOD Potions/Poisons/Wine, HQ3DMap, Aeterna Circlets, Legendary Weapons, Blacksmith's Workbech (lacks mipmaps) and High Resolution Armors, but haven't got around to processing them yet (and/or the savings are a lot less). And, in many cases, it's not an all-or-none with these. Some DXT1 textures in a mod may be saved correctly with ones that aren't. Which is why it can be a time-consuming process to optimise a mod, and why it's better if the author gets it right first time around.

Note*: Xenius' BBR -- These are deliberately uncompressed, because doing so introduces artifacts. Xenius packaged his mod perfectly. After closely examining the compressed textures in-game I couldn't see a noticeable difference and, being close to my video memory limit, decided to save memory when Khajitts and Argonians are loaded. If I had more memory I'd probably run the pack normally. I am however running XCE unchanged, as artifacts on humanoid faces are easier to see given the lower resolution of the textures.

Thanks Martigen. I've been waiting to install a big texture mod until the four gig executable is released by Bethesda.

I've tried a few of them so far and I've noticed more CTD so there must be some things about them which arent quite right.

Seems like even though people have made TES mods for years in many ways things are like starting from scratch.

Perhaps the experienced modders here could write some detailed modding guides.

Any chance of an end user guide to optimization? Since I'm sure not everyone will care enough to fix their files, I'd like to do it myself.

http://www.gamesas.com/topic/1330264-ddsopt-of-use/

Just thought I'd mention this thread to all.

I'm currently attempting a batch conversion of the entire texture folder, I suggest strongly to anyone attempting this to back up your texture folder, I won't have any results for at least another hour judging by the progress speed.

Here is a small table with all common resolutions. It should be a nice help to determine the outcome size beforehand:
http://i255.photobucket.com/albums/hh122/deathb0rnwc3/Skyrim/ResolutionsandCompression.jpg

thx for your afford! i will give it a try as soon as it is done.

I'm sure some of the experts could give a better guide, but here's my quick one based on using The Compressonator

Setting up:

• Decompress a mod (using Wyre Bash simply select 'Unpack to Project(s)')
• Browse the texture directories, open the *.dds texture files (will auto-open in The Compressonator if file association is set up) -- especially any textures where the textures and the texture's normal map (*_n.dds) are the same size. They rarely should be.
• Right-click on the texture and select 'Show Alpha' then
• Select View --> Properties from the top menu to see the current compression level and how many mipmap levels are present

Decision tree:

• If the file is the standard texture and has no alpha channel, it should be compressed with DXT1. If it isn't, compress it now (or wait and do batch-compression if you find more files)
• If the file is the standard texture and has an alpha channel, it should be compressed with DXT5. It probably is at this stage (it wouldn't contain the alpha if it was DXT1). Nothing to do -- unless it hasn't had mipmaps generated. In this case, recompress.
• If it's a normal map, it should (at least usually) contain an alpha, and the method used should be DXT3 or DXT5. If so, nothing to do -- unless it hasn't had mipmaps generated. In this case, recompress.

Compressing:

Note If not using batch, you must click 'Generate Mips' or they won't be generated. Assuming batch compression (it's just easier):

• File --> Batch Compress
• Browse to directory
• Ctrl-click select the files you want to compress. Using 'Compress All' will do all textures, including normals etc. You probably don't want this. Using 'Compress Tree' will do the same -- you can't select a filemask with these unfortunately.
• For MipMaps select 'D3DX Filter', Click Options -- select Box and Lowest Level 1x1 -- Note: Any experts want to chime in on the Dither, Mirror Pixels and Filtering options here?
  [*[ For Output Format select 'ATI 3Dc Compression', Click Options -- select DXT1 or DXT5 as required. Note: Here I also select 'Defaults RGB' and click 'Adaptive' -- from what I've seen, these are the options that help produce better quality DXT files than the Nvidia compressors.
• Click Compress!

Then back in Wyre Bash (you should be using if not!) it will pick up the changed files and you can either install the mod directly as a project, or re-compress it and replace the original archive.


That sounds excellent, thanks for letting us know!


I'd strongly suggest not doing this -- unless you're using a tool where you can selective compress different texture files with different DXT compression levels, you're going to break a whole load of textures.

I've never seen a normal with mip maps. AFAIK normals don't need them.

Normal maps are treated the same as everything else. Depending on the alpha, save as DXT3 or DXT5. No alpha = save as DXT1. DXT1 - 1 bit alpha is also a choice if you have an alpha channel with only black and white (aka no grays).

There is also another problem I've seen where textures that shouldn't have an alpha channel have an empty white one saved with it further bloating file size. Solution: delete the alpha channel and save as DXT1.


Everything is saved correctly except my LOD file, which will be fixed in an update. I saved my water surface normals uncompressed but halved the resolutions to compensate because I think they look better. 512x512 uncompressed > 1024x1024 compressed.

Fantastic, have updated the OP with this (for those that don't want to read again):

"Deathb0rn has conviniently linked a table of common texture volumes with the resulting file size after compression at various DXT levels -- this way you can usually tell what compression a texture has by its size. http://i255.photobucket.com/albums/hh122/deathb0rnwc3/Skyrim/ResolutionsandCompression.jpg The only thing to add to this is that when a texture is lacking mipmaps, it's usually 33% smaller than the sizes you see in the table."


Well there you go then. Just about every normal I've seen in texture mods has them, but if they're not needed, a good chunk of memory could be saved on every normal.


isoku! Love your mods. Thanks for the input, both of the above is news to me at least. Will update OP.

EDIT: This explains the multitude (you're right, there's loads of them) of textures I found which had white alphas that had black pixelated outlines of the parent texture. Be easier if we had a tool to automate that :/

on a seperate but similar note, has anyone checked the default textures are fully optimised? I remember it was discovered that oblivions textures were not, and they were all optimised, making a major fps increase.
Stupid things like a 2048x2048 texture for a door handle, wrong file formats, etc, etc.

Another thing to add.... Uncompressed textures.

It's been determined in the Oblivion period that DXT compression does Bad Things? to normal maps. Channels aren't compressed evenly, DXT compression artifacting causes oddities when the normal map is rendered.

To get a similar filesize savings, instead, reduce the dimensions to 1/4 its size (or half squared basically, 1024x1024 to 512x512) and save as ARGB8 Unsigned. It's uncompressed, and thus has no DXT artifacting, and the size reduction far more evenly distributes the reduction artifacting for a better looking normal map.

For an example of DXT artifacting marring a normal map, observe the vanilla nose tip on the human female races in Skyrim, or the red Pipboy buttons when viewed under the right light angle. See the weird blockiness? DXT artifacting.

And also, do remember to mipmap them.

Throttlekitty released some nVidia Texture Tools DDS Photoshop Plugin profiles with improved mipmap settings back in '07 for Oblivion modding. They're still applicable for both FO3/FONV and Skyrim textures. Check it out here: http://www.tesnexus.com/downloads/file.php?id=11817

And as a final note.... Do not upsize the texture unless you're actually adding detail to it. Larger dimensions by themselves != more detailed textures. It cannot undo size reduction detail loss.

So I discovered, fortunately I make a habit of backing up files and folders before messing around, word of warning folks, DON'T batch process!

Is there a tool that can be selective?

the only issue i have with saving as a dxtc 1 format is the fact that i don't see there being an option to save it as anything other than a 4bit per color format, meaning it's limited to essentially a 12bit color pallette. Where as the advantages of DXTC3/5 is 8 bit (24 bit color with 8 bit alpha)

So obviously if someone is making a fairly large very colorful texture package, those extra colors can make a word of difference.

Personally i have only ever saved DXTC1 when working with textures with NO alpha at all, and saved using DXTC5 with Interpolated Alpha when working with a texture with Alpha.

I know that sometimes working with Normal Maps that saving as DXTC 1 or DXTC3 can be preferred over DXTC5... but very large normal maps i don't know..

Artifacts and such aside... it's debateable in specific cases, but most cases yes dxtc1 is best or uncompressed in a smaller resolution format.

DSS Save Formats ~> http://img820.imageshack.us/img820/8342/ddsfilesaveformats.jpg