Batch generating textures from multiple psds

Post » Fri Mar 18, 2011 8:16 pm

I am in the process of updating over 1,700 loading screens for Atmospheric Loading Screens from the old http://i24.photobucket.com/albums/c3/Galahaut/Oblivion%20Loading%20Screens%20Part%202/V28.png to my new http://i285.photobucket.com/albums/ll61/Revanchism/Working/test11.png.

The process runs something like this:

1) I make the new screen by hand (I can't really automate this part and still be sure it will look good), so lets not worry about it.

2) Save the new loading screen in psd (for archive purposes) in png (for web display) and dds-dxt5 (for use in-game) in the subdirectory /Menus/Loading/.

3) Resize the screen to 50% (from 1024*512 to 512*256) and save again in dds format in /Menus50/Loading/.

Now, if you count that up, that's either over 5,100 or 6,800 extra file operations on top of the initial ones (depending how you choose to count things), not including files that will be made two times for theme packs. I'm sure you can imagine that I'd kinda prefer to save myself as many of those extra thousands of extra tedious renamings/clickings as I can. It would save errors and save me getting carpal tunnel syndrome.

So, I was thinking I could maybe make use of Irfanview's command line capability to perform some batch renaming and resizing. The problem is, it seems like dds format is not supported in the batch commands, so that's something like >3,400 file operations I can't use it for. I mean, I can use it for batch generating the pngs and also resizing into, say bmps that I can then re-format one-by-one, but that's only saved me half my "excess" work.

Does anyone have any suggestions/scripts to help me out here? Remaking 1,700 loading screens is a large enough job as it is, particularly with my poor skills.

Hoping desperately, :cookie:

Vac
User avatar
Jynx Anthropic
 
Posts: 3352
Joined: Fri Sep 08, 2006 9:36 pm

Post » Fri Mar 18, 2011 11:51 am

I am in the process of updating over 1,700 loading screens for Atmospheric Loading Screens from the old http://i24.photobucket.com/albums/c3/Galahaut/Oblivion%20Loading%20Screens%20Part%202/V28.png to my new http://i285.photobucket.com/albums/ll61/Revanchism/Working/test11.png.

The process runs something like this:

1) I make the new screen by hand (I can't really automate this part and still be sure it will look good), so lets not worry about it.

2) Save the new loading screen in psd (for archive purposes) in png (for web display) and dds-dxt5 (for use in-game) in the subdirectory /Menus/Loading/.

3) Resize the screen to 50% (from 1024*512 to 512*256) and save again in dds format in /Menus50/Loading/.

Now, if you count that up, that's either over 5,100 or 6,800 extra file operations on top of the initial ones (depending how you choose to count things), not including files that will be made two times for theme packs. I'm sure you can imagine that I'd kinda prefer to save myself as many of those extra thousands of extra tedious renamings/clickings as I can. It would save errors and save me getting carpal tunnel syndrome.

So, I was thinking I could maybe make use of Irfanview's command line capability to perform some batch renaming and resizing. The problem is, it seems like dds format is not supported in the batch commands, so that's something like >3,400 file operations I can't use it for. I mean, I can use it for batch generating the pngs and also resizing into, say bmps that I can then re-format one-by-one, but that's only saved me half my "excess" work.

Does anyone have any suggestions/scripts to help me out here? Remaking 1,700 loading screens is a large enough job as it is, particularly with my poor skills.

Hoping desperately, :cookie:

Vac


OK, I might have something for you. That is if you're using Photoshop and you can save .dds files in there (with the Nvidia plugin, for example - only works up to Photoshop CS4 though).

That said, here's what you do:
Spoiler

1. Open one file
2. Go to window -> actions
3. press the "new action button" (it's on the bottom, next to the "bin" icon)
4. name it something specific like "loading screens process 01"
5. press the "record" button (circle icon)

6. file -> save as -> *doesn't matter where you save it as long as it's a different folder than the current one* -> psd -> tick "as a copy"
7. file -> save as -> *same as above, can be the same folder as the previous one though* -> png -> tick "as a copy"
8. press "stop" button again.

9. repeat steps 3, 4, 5
10. file -> save as -> *don't matter* -> dds -> tick "as a copy" -> set the options in the dialogue accordingly (hopefully, this will save the settings for further use, I can't test as I don't have the plugin. If it doesn't, then you might have to click through them for every file later)
11. press "stop" button

12. repeat steps 3, 4, 5
13. Go to image -> image size -> change dimensions to "percent" and enter 50 in both boxes. Choose "Bicubic Sharper" at the bottom (that's the one I recommend, you could leave "bicubic"if you think it looks better).
14. file -> save as -> *don't matter* -> dds -> -> tick "as a copy" -> set the options
15. press "stop button

16. make sure all the files you want processed are together in one folder
17. Go to file -> automate -> batch
18. select "loading screens process 01" for "action"
19. select "folder" for "source"
20. click choose and select the folder where the images are at the moment (tick "include subfolders, if it applies, you might need to select "supress file open options dialogs" if that comes up when you open a loading screen file)
21. select "folder" for "destination" (make sure it's different from the "source" one)
22. clock choose and select the folder where you want your psd and png files saved
23. tick "override action "save as commands"
24. file naming: I'd leave "document name + extension" if you need you can rename them later with IrfanView.
25. click "ok" and it will go through the files

26. repeat 17 - 25 for "loading screens process 02" (second action). At "22" select the "menus/loading" folder.
27. repeat 17 - 25 for "loading screens process 03" (third action). At "22" select the "menus50/loading" folder.


I know it looks like a lot, but it's quite simple. Take it a step at a time. Let me know if there are any problems.

Cheers!

cc
User avatar
Monika Fiolek
 
Posts: 3472
Joined: Tue Jun 20, 2006 6:57 pm

Post » Fri Mar 18, 2011 12:31 pm

OK, I might have something for you. That is if you're using Photoshop and you can save .dds files in there (with the Nvidia plugin, for example - only works up to Photoshop CS4 though).
Ahh, nuts, I knew I'd forget to include some relevant information. Myapologies.

A short history of my troubles with the dds format:

I have CS4, but the NVidia plugins consistently fail to work. My previous attempt to get help with this garnered a response along the lines of "did you install it to the right directory?", "yes, the dds.8bi file shows up in the \Adobe Photoshop CS4\Plug-ins\File Formats\ directory and Photoshop is installed outside of the default Program Files directory, so UAC shouldn't be an issue", "Oh, ok, I don't know then, hope you find a fix", which of course I didn't. Photoshop remains consistently unable to load or save dds files.

I have the Gimp, and can save dds files, as long as they're uncompressed. Compressed dds files get noticeable and problematic artifacts. I've tried reinstalling the program completely, rolling back to previous versions of Gimp and so forth. Previous attempts at getting help with this consisted of forcing me to repeat the same steps I had done myself before I asked for help and eventually came to an "Oh, ok, I don't know then, hope you find a fix".

I have Paint.Net which is the only method I have that I know will save compressed dds files without stupid artifacts. No problem with this program except the annoyingly long-winded interface.

If you've got any bright ideas why the NVidia plugins don't work for me and/or why I get artifacts with Gimp then please, help me out with that, otherwise I need a solution that doesn't require Photoshop or Gimp for the dds portions.

Vac

Edit: And before you ask, yes, I installed the NVidia plugins to the 32-bit version of Photoshop.
User avatar
Tarka
 
Posts: 3430
Joined: Sun Jun 10, 2007 9:22 pm

Post » Fri Mar 18, 2011 9:13 am

snip


I'm not sure what you already tried. A few basic ideas that come to mind:
1. Try another version of Photoshop
2. Try another version of the plugin (the website has both 8.23 and 7.83)
3. Alternatively to 1 and 2, try it another computer (your photoshop, your copy of the plugin). You could test this one someone running the same version of windows and/or another.

As I said, it's basic stuff, but just mentioned it in case you skipped one of them.

Alternatively, there's AMD's http://developer.amd.com/gpu/compressonator/Pages/default.aspx. There's a tutorial http://www.katsbits.com/tutorials/textures/making-dds-using-ati-compressonator.php for how to use it. It doesn't cover batch converting of files, but that should be pretty easy to do. You can take a look at that and see if the conversion algorithm is as good as whatever it is you're using now. If the images look worse, you may decide to look for another option. If you like the result, however, in the steps above save the files as tga.

Of course, I ought to mention that in order for this to work, your starting images should be open-able in Photoshop. Converting dds -> tga -> dds will double the artifacts and it's not worth it.

Hope you sort your nVidia plugin out or that you've got the originals for the loading screens (i.e. psd or something).

cc
User avatar
Tamara Dost
 
Posts: 3445
Joined: Mon Mar 12, 2007 12:20 pm

Post » Fri Mar 18, 2011 1:59 pm

I'm not sure what you already tried. A few basic ideas that come to mind:
1. Try another version of Photoshop
2. Try another version of the plugin (the website has both 8.23 and 7.83)
3. Alternatively to 1 and 2, try it another computer (your photoshop, your copy of the plugin). You could test this one someone running the same version of windows and/or another.

1. I have a license for this version, CS4.
2. Done, neither work.
3. Not that easy to do as I don't have any real access to any other computer other than one at work. Even if I try it out and it works, it's still no practical use to me.

I'll take a look at the Compressionator and see how that works out...

Vac
User avatar
Adriana Lenzo
 
Posts: 3446
Joined: Tue Apr 03, 2007 1:32 am

Post » Fri Mar 18, 2011 1:47 pm

OK, so I've tested the Compressonator, and it's not generating any obvious artifacts, so hopefully won't cause me to get people complaining of complete noise on their systems (don't you love when things work ok for one system and complete garbage for another?)
It doesn't cover batch converting of files, but that should be pretty easy to do.
Indeed, a little digging through the menus and it seems to work just fine. Strangely won't allow me to output to a different directory: despite supposedly providing that option, the "Compress" button remains firmly greyed out unless I select the option to keep in the same directory.

So my workflow is now going to be something like the following:

1) Make the loading screens and save them as psds.

2) Batch convert the psds to an uncompressed (i.e. lossless) format such as bmp using Irfanview.

3) Batch convert the psds a second time to bmps at 50% size for the Menus50 copies using Irfanview.

4) Batch convert the psds a third time to pngs for the online gallery using Irfanview.

5) Batch convert the full size bmps to dds dxt5 format using The Compressonator.

6) Batch convert the half size bmps to dds dxt5 format using the Compressonator.

My life would be *sooooo* much easier if I could get the plugins for Photoshop to just work, but I now have a sensible solution.

Five batch conversion jobs may be unnecessarily extra work, but it's still a vast reduction to doing literally thousands of conversions individually.

Thanks for the pointer to the Compressonator cc, it is *greatly appreciated*, and means I am *much* more likely to make progress with ALS. You have no idea how depressing it was to be stymied on working on ALS because it was so difficult/inconvenient to handle the texture compression.

Of course, if anyone has any other suggestions as to how I can fix the Photoshop plugins, please, I'd be more than happy to cut my 6 steps down to 5 and use one less program.

Vac

Edit: A little backgound reading and I discover that pngs are in fact lossless, so it seems I can use them as the intermediary file between the psds and the textures saving a step above.

Edit2: Huh. Unfortunately, the compressed dds files generated by the Compressonator are visibly lower quality than the dds files generated by the Gimp (sans aforementioned artifacts). I'm kind of stuck with this as it is the only option I have for mass-producing textures, but I'd seriously appreciate assistance still as I obviously don't *like* having poorer quality textures.
User avatar
Danielle Brown
 
Posts: 3380
Joined: Wed Sep 27, 2006 6:03 am

Post » Fri Mar 18, 2011 8:34 pm

Vac,

I sent you a PM regarding your Photoshop plugin issue. Hopefully it'll help.
User avatar
mimi_lys
 
Posts: 3514
Joined: Mon Apr 09, 2007 11:17 am

Post » Fri Mar 18, 2011 9:26 pm

Please let me know if you get your Photoshop plugin sorted - I'd like to know how you did it for future reference!

cc
User avatar
Darlene Delk
 
Posts: 3413
Joined: Mon Aug 27, 2007 3:48 am

Post » Fri Mar 18, 2011 11:29 am

No solution at present, Hanaisse suggested reinstalling Photoshop to the default directory and also doing some registry hacks. I'll summarise:

  • I still can't use Photoshop to open or save textures.
  • The Gimp will open and save textures just fine, but will corrupt textures when saving with DXT5 format (I should probably check the other compression formats and see if it's the same for them).
  • Paint.net works just fine for opening and saving textures, but it's just painfully slow to work through the saving interface, so not useful for thousands of jobs.
  • Irfanview doesn't handle the dds format for batch jobs (it appears to work ok for regular manual work though).
  • The Compressonator functions just fine, but there's a greater loss of quality when saving into compressed dds format compared to some of the other tools (that don't function properly/usefully).


:brokencomputer: :brokencomputer: :brokencomputer:
:brokencomputer: :brokencomputer:
:brokencomputer:

On the basis that I'll be retaining archive copies, and so can repeat the mass-conversion job again at a future point if another, better option comes along, I shall go with the Compressonator for now. Realistically, it is the only choice other than giving up or spending weeks and weeks doing it by hand. Uhh, so the only choice. If another batch/automated option pops up which offers me a better quality solution, I'll simply do it again (probably swearing about it, mind).

Vac (is still vaguely hopeful that someone, anyone might be able to help...?)
User avatar
OnlyDumazzapplyhere
 
Posts: 3445
Joined: Wed Jan 24, 2007 12:43 am

Post » Fri Mar 18, 2011 11:08 am

I'll provide some sympathy and encouragement, and join you in your hopeful state. :hugs:
User avatar
DarkGypsy
 
Posts: 3309
Joined: Tue Jan 23, 2007 11:32 am

Post » Fri Mar 18, 2011 3:43 pm

A few other ideas:

1. Not sure if you know about this, if it does any sort of batch processing or if it help in any way, but there's a Paint.Net plugin called http://forums.getpaint.net/index.php?/topic/19644-tool-for-paintnet-pdn2dds/

2. If you're not using Vista or Windows 7, you can try the http://www.tesnexus.com/downloads/file.php?id=8698.

3. This might not work, as it's from nVidia as well, but it's a command line tool: http://developer.nvidia.com/object/dds_utilities_legacy.html

For this latter, it comes with a pdf explaining the various options. If you have problems with using it, let me know. I'm not sure what your background is in command-line tools :)

A short guide though:

Install the tools somewhere easily accessible like c:\dds utilities

1. go under start -> run (alternatively, press the windows button + r)
2. type cmd
3. use "cd.." (without the quotes), to move outside of a directory, "cd " to move into a directory. For example, getting from c:\users\vacuity to c:\dds utilities you'd type cd.. -> enter -> cd.. -> enter (now you're in c:\) -> cd dds utilities
4. type the name of the exe file: "nvdxt.exe". It shows a list of options, they are in the pdf as well.
5. Here is a sample line a wrote:
nvdxt -file e:\documents\test\*.tga -outdir e:\documents\testOut -dxt1a -nmips 8 -dither -quality_highest
change the input directory path with yours, the output with yours. if you want dxt5 instead, use "-dxt5" instead of "-dxt1a". There's also "-dxt1c" which you might need, or "-dxt3" which is the default. "*.tga" means all tga files in that folder. If your images are currently saved as psd, type "*.psd" instead.

Finally, you might be able to smooth things out a bit in Paint.Net afterwards, but it's going to take some extra time. I'm not sure if it has a "Reduce Noise" filter, that would help. Also, if you like the look and you have one available, you can try adding a noise layer with something like 5% noise. It will make your image grainy-er but the conversion artefacts will be lost. This part would be much easier to do in Photoshop, naturally, so if you make the mod and want something like this done, ask someone else to tweak the files for you. I would offer, but I can't use the nVidia plugin either.

Keep me posted please :)

cc
User avatar
Tiffany Castillo
 
Posts: 3429
Joined: Mon Oct 22, 2007 7:09 am

Post » Fri Mar 18, 2011 12:11 pm

A few other ideas:
Thankyou for looking on my behalf while I am making loading screens. :)
1. Not sure if you know about this, if it does any sort of batch processing or if it help in any way, but there's a Paint.Net plugin called http://forums.getpaint.net/index.php?/topic/19644-tool-for-paintnet-pdn2dds/
Looks like I'd have to write my own batch files to use that. Not impossible, but not overly convenient.
2. If you're not using Vista or Windows 7, you can try the http://www.tesnexus.com/downloads/file.php?id=8698.
I am on Win7, and I'm not much in the mood for trying my hand at more frustration right now. I may check it out later when I'm feeling a little less put out by tools not working as desired.
3. This might not work, as it's from nVidia as well, but it's a command line tool: http://developer.nvidia.com/object/dds_utilities_legacy.html
That actually looks quite useful now I look at it more carefully.
Also, if you like the look and you have one available, you can try adding a noise layer with something like 5% noise. It will make your image grainy-er but the conversion artefacts will be lost.
One or other of us is confused (or perhaps both). Have you looked at the ALS loading screens before? There is no purpose to add a noise filter to a finished loading screen. Besides, what I mean by artifacts is corruption, not simply poor quality. If you had used the first release I made after taking over development of ALS, you would likely have seen one or other of the problems that Gimp caused in my compressed textures. Some people simply get small, odd-coloured blocks here and there in the texture when it's displayed in-game (and this is how the compressed textures look in the Gimp); in some textures they're hardly noticeable, in others they're quite noticeable. For other users, the entire loading screen was displayed as something similar to white noise in-game.

Vac
User avatar
Misty lt
 
Posts: 3400
Joined: Mon Dec 25, 2006 10:06 am

Post » Fri Mar 18, 2011 12:31 pm

The Compressonator functions just fine, but there's a greater loss of quality when saving into compressed dds format compared to some of the other tools (that don't function properly/usefully).


A little off topic...but I have (belatedly) noticed the same issue with the Compressonator...as well as having problems with icons I compressed with the Compressonator having wonky alpha maps that caused my icons to be incompatible with medium and small texture game settings. DDS Converter seems to do the best/most reliable job of compressing files to DDS format (at least among the tools I presently have access to), but I can't get DDS Converter to function properly in a Windows 7 environment. (So I was forced to use my old rig and DDS Converter to compress my textures.)

Regards,

Hem
User avatar
Steve Bates
 
Posts: 3447
Joined: Sun Aug 26, 2007 2:51 pm

Post » Fri Mar 18, 2011 8:23 pm

3. This might not work, as it's from nVidia as well, but it's a command line tool: http://developer.nvidia.com/object/dds_utilities_legacy.html
Well, a quick report on my further investigation of this and also their http://developer.nvidia.com/object/texture_tools.html.

Despite NVidia billing Texture Tools 2 as CUDA accelerated, I couldn't find anything that said it wouldn't work without a CUDA card (I'm on an AMD card right now). So I tried that out first, and I have to say, I was *very* impressed. The resulting compressed texture was very good quality; probably the best I've seen thus far. The problems were that it didn't recognise png format (despite this supposedly being supported) (I just used my psds, so no major inconcenience), the compression utility doesn't appear to support wildcards in filenames, thus making it inconvenient to do batch jobs (I'd need to create a batch file), and tied to this, a distinct lack of options (or at least documented options). This seems to be geared more towards developers including the libraries in their own software, rather than as a set of stand-alone tools.

So I took a look at the legacy tools. The documentation is nice and comprehensive (as long as you're not scared of command lines), there are plenty of options (it's got to be said, if you don't know much about texture formats, this would probably be intimidating), but the "production" quality was definitely not up to the par of their Texture Tools 2. I'm re-running right now with "Highest" quality, which is noticeably slower, but the quality seems roughly comparable to the quality Texture Tools 2 was giving me as standard. Interestingly, despite being "slower" than Texture Tools 2, this is still considerably faster than the Compressonator, and with vastly superior quality. As I can use wildcards for the input files of the compression program this precludes any major inconvenience with doing batch jobs.

The current conclusion is that the legacy texture tools are the overall winner when taking into account quality, convenience and speed. The quality on "highest" is good, the speed on "highest" is acceptable on a modern CPU, and doing batch jobs is fairly simple as long as you can handle command line utilities comfortably.

So... Unless and until someone re-compiles the Texture Tools 2 libraries into a tool with more options and easy support for batch jobs (http://www.youtube.com/watch?v=TVH3IBr_Ipk...), I'll stick with this solution for mass generating my loading screen textures. It's still a blasted inconvenience to not be able to load or save textures in Photoshop, or save compressed textures in Gimp. :down:

Vac
User avatar
Ridhwan Hemsome
 
Posts: 3501
Joined: Sun May 06, 2007 2:13 pm


Return to IV - Oblivion