RAMDisk - Load BSA Archives From (or loose files) RAM

This is for the modder extremist who is looking for the best possible performance.
Below is the link to the product.

http://memory.dataram.com/products-and-services/software/ramdisk

English PDF - http://memory.dataram.com/__documents/file/Dataram_User_Manual_35.pdf

It's been determined that this product allows a user to allocate un-used RAM for storage use.
A drive letter is assigned and the RAM can be access just like a regular HDD. Ideally, this is for players with 6 to 8 gb of RAM and beyond using a 64bit OS. However, if you are a 32bit OS user and have more than 4gb of ram you can still benefit from this application. (Though, not sure why anyone would run a 32bit OS with more than 4gb of ram.)

Below are some insightful tests. I figured if a portion of my RAM was basically now a storage device, then I should be able to test it as such. Below are HD TACH tests I performed on a 150gb Veloci Raptor vs. 100gb Inferno SSD vs. 100gb RevoDrive X2 vs. 4gb 1333mhz Viper series RAM... YOU WILL BE AMAZED by the results.

http://i295.photobucket.com/albums/mm150/far327/150gbVelociRaptorHDDtest.jpg Avg Read = 105.8mbps
http://i295.photobucket.com/albums/mm150/far327/100gbInfernoSSDtest.jpg Avg Read = 170.2mbps
http://i295.photobucket.com/albums/mm150/far327/RevoX2results.jpg Avg Read = 240.1mbps
http://i295.photobucket.com/albums/mm150/far327/4gb1333mhzPatiotVibichMtest.jpg Avg Read = 731.0mbps

This would benefit Oblivion players by allowing them to load .bsa archives (or loose files) from RAMDisk!
Ideally, loading large overhauls from RAMDisk such as QTP3, Better Cities etc... should result in
faster load times and less stutter do to I/O file swapping. Mechanical HDD owners will notice the greatest improvement while Solid State Drive owners will notice some improvement.

http://www.megaupload.com/?d=F1J7Y1WR

http://www.gamesas.com/index.php?/topic/1174085-anyone-try-ramdisk-before/page__view__findpost__p__17469935 posted by kandiedan. He has found a way to hardlink various drive paths so that Oblivion can read files from multiple file locations.

And a shorter version of the same instructions here...

Only if you never restart or turn your computer off. You'd lose everything the moment power went out if you decided to put your entire /Oblivion/ folder on a RAMdisk.

Also, I don't see how this is supposed to be a solution for people with inadequate memory. This can be a solution for people with lots of memory - use the RAMdisk as your swap file, and you'll see improvement.

The linked software avoids this.


Neither do I!

If I have a SSD, does this even matter to me?

I'd be hard pressed to fit my 30GB install into a RAM disk of any sort

This is probably better utilized as a way to trick Windows into using it for the virtual memory system instead.

Frankly though, I don't see this as being of any value at all on 32 bit systems and only of marginal benefit on 64 bit systems with less than 8GB.

I was wrong in saying this would help with low memory users. I think I had this confused with another product. I am translating Japanese Oblivion blogs, so bare with me.

One thing I learned about using RAMDisk which is a great benefit... If you decided to load your Meshes, Textures .bsa files or other large .bsa files from your physical RAM you would see some dramatic I/O improvements!
I also run Oblivion off a SSD currently, but to be honest, my RAM is still faster. Trust me when I say, you will be amazed when you simply copy your Texture.bsa file to your RAM!!! The write mbps is unreal!
I am used to seeing very high write speeds when copying files within my SSD, but not as fast as my RAM. And not as consistantly fast as my RAM. Download the 4gb trial and test it. IF you have an HDD, you will see a HUGE difference in I/O speed.

I'm thinking I will load all of my texture mods thru RAM now that I have have 8gb. This should allow me a little space to experiment with BUT if you had say 12gb or even 16gb... OMG!!! Say goodbye to stutter due to I/O bottlenecking.

EDIT: The write speed when I copied my meshes.bsa file from my SSD to my RAM was avg 250mbps. Something interesting though... I deleted the file and re-copied it... Speed went to 1.15gbps!! Sooo, this has something to do with RAM caching.
Wonder if the caching would benefit for Oblivion ,bsa files when packing/unpacking?
It's clear that my copying files from my SSD to RAM was not showing the full potential in which my RAM can perform read/write functions. This might sound weird, but my SSD is actually bottlenecking my RAM!

What kind of lag at boot is there when it has to copy stuff from the drive back into the RAM drive? Surely there has to be some since that RAM has to be populated before it can be used?

Also wouldn't copying the Meshes file be of more benefit (considering such mods as Open Cities Reborn, etc.) ? I only have 6GB of RAM, but this does sound fun haha.

RAM disk is only really practical on 64 bit systems, with at LEAST 8 gig of ram. Any less, and even putting your swap file there would still have you paging to the hard disk fairly often.

This WAS something I had considered at one time...... as it would be a HUGE leap in performance, IF you could load your entire install into it. Trouble with that is..... my install is over 20gig, and the MOST memory I have seen supported on PC motherboards is about 26 gig....... I don't think copying all that data to a RAMDrive would be exactly speedy either..... Not to mention, that much RAM would be HIDEOUSLY expensive. SSD is cheaper, and not much slower.....

Just tried that this morning. There is a long delay during the Windows7 logo. However, I believe there is a setting to by pass where it happens during startup. You would then have to manually load the save data from within the RAMDisk app after windows loads.

@Venom8 - I did texture files first since there are more files. Per every single mesh there are many times multiple textures. The more files your I/O device has to browse, the more chance of stutter, or at least that makes sense to me.
If I had more RAM, I would certainly add the meshes.bsa too.

@Hey You - I admit it wasn't very practical in suggesting to load all of Oblivion from RAM. Perhaps in the future maybe yes, but by then SSD will probably be as fast. But, loading .bsa files from your RAM is certainly worth a look. TRUST ME!

I did some testing this morning. Figured if my RAM was basically now a storage device, then I should be able to test it as such. This is a HD TACH test on my 150gb Veloci Raptor vs. 100gb Inferno SSD vs. 4gb 1333mhz Viper series RAM... YOU WILL BE AMAZED by the results.

http://i295.photobucket.com/albums/mm150/far327/150gbVelociRaptorHDDtest.jpg
http://i295.photobucket.com/albums/mm150/far327/100gbInfernoSSDtest.jpg
http://i295.photobucket.com/albums/mm150/far327/4gb1333mhzPatiotVibichMtest.jpg

A more fair test would be each drive with the same storage space. Keep in mind that each device contained data.
But just look at the Avg read speed differences and the difference in Bust Speed.
Only thing that concerns me is the CPU utilization when using RAM as the storage device. I'll have to see if this CPU usage shows the same when playing Oblivion.

Holy smoke....... but, I suppose, it shouldn't come as that much of a surprise. RAM has access to the system bus, and doesn't need to hit the PCI bus at all. Of course it will be faster.

Question though, how did you manage to get oblivion to look for the bsa's in a folder OTHER than the install folder?

I got the info from this site - http://sites.google.com/site/ballofflame/theoblivionperformanceproject From this post by Skeleton -

So as an example if you were loading your Oblivion - Meshes.bsa from your RAMDisk with a drive letter of R:/

You would change this line "SMasterMeshesArchiveFileName=Oblivion - Meshes.bsa"

to look like this "SMasterMeshesArchiveFileName=R:/Oblivion - Meshes.bsa"

Additionally, you would need to change the "Oblivion - Meshes.bsa" line in the SArchiveList= of your .ini to "R:/Oblivion - Meshes.bsa"


I feel like all my stuttering is gone with exception to AI loading. I wonder if it would make any difference on AI loading to have the files which process the AI run from RAMDisk as well? Anyone have any thoughts on this?

EDIT: Earlier I mentioned that copying my meshes.bsa from SSD to RAMDisk ran at about 250mbps. Just want to point out that my SSD was actually bottlenecking the RAMDisks performance. If you were loading your .bsa files from RAMDisk, would this mean the bottleneck would be eliminated or is there still fileswap between the Oblivion install location and the RAMDisk for temp unpacking? Either way, I am seeing a large reduction in stutter.

RAMDISK is nothing new it has been discussed a few time in the past.
Oblivion game stutter is always the result of the Hard-Drive transferring files to the system memory.

There are 2 problems on a 32bit system with RAMDISK:
(only 3GB free space for the game in memory)

1# Modded Oblivion is simply to big to put into system memory.
(only 3GB free space for the game in memory, My game goes to +25GB of data, LOL)

2# Modded Oblivion will exceed the GPU memory available.
(only 1 to 3GB GPU memory available, the rest will be stored in system memory)

The RAMDISK aspect does work to some extent, but I don’t seem to gain any extra result.
I guess thats what happens when you push the game to its limit !


PS: And now I am messing it up even more by using the new OBGE Shaders.

- I think I have already back tracked my statements about loading a full Oblivion install into RAMDisk. No need to keep pointing this out. I am only suggesting RAMDisk could be used for loading .BSA files as opposed the installation directory. If someone had extra RAM not getting utilized, why not make good use of it?

- I don't understand what you meant by "modded Oblivion exceeding GPU memory available"? Not sure what that has to do with RAMDisk.

- Based on the tests I've done, I think there is good reason to give RAMDisk a shot. You can't deny those HD TACH results.

- OBGE Shaders? Again, I am not following what that has to do with RAMDisk.

Unfortunately in the modded state my game is in I couldn't even fit the BSA files into the ramdisk. They exceed the 4GB trial size. Heavily modded games are probably still better off using an SSD because that's going to be a phenomenal speed difference compared to a mechanical drive.

Yeah, i had to compress my QTP3+Improved Flora/Trees/Doors textures .bsa files by 20% just to get them to fit into the 4gb trial size limit. I will probably drop $15 on the application once I get more RAM for my next build, but even still the performance is solid. At the very least, just for fun you should just try copying a large file to RAMDIsk and then copy the same file to ram disk replacing the original. You should see a jump in speed due to caching. You'll get something like 1.15gbps transfer rate! Or just test with HD TACH like I did. The results are stunning!

So once I am done playing Oblivion, how do I "unload" the loaded bsa files from RAM? Maybe this is something mentioned in the manual, but just wanted to confirm if this is fairly easy to do.

This looks interesting, and I may just try it.

@far327: can you post a step-by-step guide to what you did to use this utility with Oblivion? Maybe post that in the OP as well...

RAM disk, brings back the old DOS days.

Regardless, to copy the files you need to it will take time, still copying from HD to RAM. Don't really need to save the data as the saves are on the HD already. Unless you have lag and stuttering, why even bother. Except maybe because you can

- The RAMDisk software utility has a Start (Load) Stop (Unload) button. You have the option to Save the data stored in RAMDisk as an image file so that way when you reboot, you can simply load that image file again. One thing I did notice is the time to load/unload is somewhat lengthy. I have almost a 4gb image that i load into RAMDisk which probably takes a solid 60 seconds to load/unload.

- I can work on a guide. I will try to today since work is slow here.

Wanted to point out this feature from the PDF file...

That claim doesn't ring true somehow. Given that 32 bit architecture can't address memory beyond 4GB I have to wonder how they're getting away with claims of being able to do something Windows itself will not allow.

Yeah, I'm doubtful on this one as well... But won't know till we try.
I finished a rough draft tutorial on how to make Oblivion read .bsa files off of a created RAMDisk. It is linked on the OP and in .rtf format.

EDIT: I have updated the tutorial to be smaller and more simplified. Please re-download if you recently grabbed it.

I understand this could be of great use if I put my meshes and textures BSAs into the ramdisk and redirected through oblivion.ini.... if I was using vanilla Oblivion. How much use will it be if I'm using a heavily modded install including MMM, body mods, QTP3, Better Cities, pyffied meshes, and much more. How much of the data in the two big BSAs is the game still accessing, really?

Sorry I missed your post. It's not a popular topic and comments get buried pretty quickly.

Well, the REAL benefit comes in when you are using heavy mods like QTP3 and Better Cities etc... .bsa Large mods like those tend to add more stutter because of the larger files sizes used for textures and the additional on screen objects needing to be rendered. Loading these files from RAM would speed up the loading/unloading process, or I/O of these files.

Files should not to exceed 2gb, so you have to create multiple .bsa files. You can add .bsa name to your Oblivion.ini so that they'll load, or you can create a dummy .esp file to force .bsa files to load. Just have to make sure the .esp file matches whatever you name the .bsa.

I haven't gotten to far into using RAMDisk to load mods such as Better Cities though. BC has it's own .bsa file. I know there is a work around to get it working, but I haven't spent the time on it yet. I will soon though as I just upgraded to 16gb of ram, so I have some room to play with.

BTW, I added another test result with a 100gb RevoDrive X2 on the OP. Should give people a good idea where performance stands with HDD/SDD/REVO/RAM

http://i295.photobucket.com/albums/mm150/far327/150gbVelociRaptorHDDtest.jpg Avg Read = 105.8mbps
http://i295.photobucket.com/albums/mm150/far327/100gbInfernoSSDtest.jpg Avg Read = 170.2mbps
http://i295.photobucket.com/albums/mm150/far327/RevoX2results.jpg Avg Read = 240.1mbps
http://i295.photobucket.com/albums/mm150/far327/4gb1333mhzPatiotVibichMtest.jpg Avg Read = 731.0mbps

So, I tried this a while ago.

I run a heavily modded oblivion . . .
QTP3R
FCOM
Better Cities
AEVWD
and a whole lot more . . .

System Setup
E6750 Overclocked to 3.6 GHz
ASRock P45DE3 - 16 GB 1333 RAM
4870 1GB
30 GB Vertex SSD
Win 7 64bit
1680x1050 resolution monitor

First background information -
FPS on this system is limited by the CPU. Anywhere that I was getting low FPS (mainly in cities, but some countryside), change the CPU directly affected the FPS. Decreasing the processor 20% would result in a 20% decrease in frame rate. Other changes to to FSB or Memory (that didn't affect the core clock) had no measurable effect on frame rate. Moving to a SSD/Ram Drive will NOT increase your FPS, just decrease loading times

Second -
Oblivion Stutter Remover - It helped me a lot when i was running on a HDD. It was installed for all my tests. Not sure if it helps, but it's still there.

Test 1 - Moving Oblivion Install from HDD->SSD
I noticed a significant improvement in load times and less "stuttering". However the "stuttering" when changing cells in the countryside was still "noticeable". Imagine the frame rate dropping from 40ish down to 10-15 ish for a couple of frames.

Test 2 - Moving Data Folder from SSD->RAMDisk
Now I don't have the RAM to move 30GB to the RAMDisk, so only some files were moved (I'll discuss this in a bit).
Area Load Times (whoa!!!! maybe twice as fast). I rarely notice stuttering when changing cells in the country side. My system is already working pretty hard to render all of the AEVWD objects so frame rate varies quite a bit in the countryside, but the cell change "stutter" effect is not an annoyance anymore.


RAM Drive Setup.
Assumptions I made because I don't know how the Oblivion engine reads files from BSAs.
1) BSAs for sounds/voice ect. These go on the SSD or the HDD. Why?? Don't need to load them when changing cells.
2) BSAs for other worldspaces (bartholom, SI, ect) need to go on the SSD. When I spend a lot of time in SI, I swap them with the Oblivion Bsas on the Ram Drive
3) Textures/Meshes that are commonly encountered when running around in the countryside need to be on the RAM drive. (I have base Meshes/Textures from + BASE/OOO/MMM/FCOM/UL ect on the RAM Drive)
4) All of the loose files in "data" folder need to be on the RAM drive as the engine will need to figure out where to load the files from (BSA or data folder).


My steps . . .
SSD -> C: Drive
RAM Drive -> Z: Drive approx 11GB

1) Install Oblivion on the SSD

2) Move the Data folder to "Z:\Od" Use a NTFS hard link to link "C:\Oblivion\Data" -> "Z:\Od"

3) Move non-critical BSAs to C:\Bs\ and link with "oblivion.ini". Note 255 Character limit, so rename BSAs to VERY SHORT NAMES and keep the path names short.

4) Move noncritical folders (music,videos,ect) back to "C:" Drive and link with NTFS hard links

5) Repack Meshes/Textures BSAs with QTPS, UOP,ect included and compressed. Link the resulting BSAs with "oblivion.ini"
I want to fit as much of the "core" meshes/textures onto the RAM drive without wasting space. Therefore I unpacked the base Meshes and Textures BSAs, applied all of my "known overrides": QTPS, B&M, UOP, ect on top of those. Then repacked compressed BSAs. You can save about a 1 GB of space on your RAM drive because you don't have duplicate files.

6) Install all of my pre-configured mods with Wryebash (I love bain)

7) Create a RAM drive image to load when I want to play Oblivion.

8) Create a Backup RAM drive image for when the primary image gets corrupted

9) Remake the Images whenever I change anything in the data folder. (If you forget, Bain does a decent job of telling you what files are missing that it thinks should be there).