Why Are My Save Files So Big?

Post » Sun Nov 28, 2010 8:34 am

Recently my game load and save times have become slow, in the order of minutes. I decided that the amount of clutter I was storing as a result of not being able to sell all my collected loot to merchants might be the problem, so I set about selling as much as I could and disposing of items from quests I had put on one side "just in case"/ a museum of my exploits. Even resorted to using the waste bin to empty containers and furniture.

As a result of all this the save file size is dropping now 16Mb from 20Mb. But the .OBSE file (i'm running pluggy 1.32), just keeps growing. It's now 15Mb and it seems to growing all the time (slowly).

Does any one have any idea how to reduce the file size, if it is at all possible?

Are the any utilities that might allow me to see what is going in to this file?

My loadorder has been relatively stable for quite sometime so it's not that I've added some major overhaul. The only changes are updates to the mods currently in use and updating Bash patch and swapping out Living Economy for Enhanced Economy.

Currently running OBSE latest beta.
User avatar
Alyna
 
Posts: 3412
Joined: Wed Aug 30, 2006 4:54 am

Post » Sun Nov 28, 2010 7:30 pm

Post your mod list so people can check if there are any known culprits.

Include version numbers if possible.
User avatar
Kay O'Hara
 
Posts: 3366
Joined: Sun Jan 14, 2007 8:04 pm

Post » Sun Nov 28, 2010 11:44 am

Post your mod list so people can check if there are any known culprits.

Include version numbers if possible.


Active Mod Files:00  Oblivion.esm01  Cobl Main.esm  [Version 1.72]02  Oscuro's_Oblivion_Overhaul.esm  [Version 1.34Beta]03  Mart's Monster Mod.esm  [Version 3.5.5b4]04  Mart's Monster Mod for OOO.esm  [Version 0.9.8b5]05  Qarls_Harvest.esm06  TamrielTravellers.esm  [Version 1.36]07  Kvatch Rebuilt.esm08  VVAdvanced.esm  [Version 1.00]09  CM Partners.esm0A  Progress.esm  [Version 2.2]0B  Windfall.esm==  ROOT=====================0C  DLCBattlehornCastle.esp0D  DLCBattlehornCastle - Unofficial Patch.esp  [Version 1.0.4]0E  Streamline 3.1.esp0F  Unofficial Oblivion Patch.esp  [Version 3.2.4]==  LIBRARY==================10  Immediate Character Generation.esp11  nGCD.esp12  nGCD Birthsigns.esp13  nGCD Oghma Infinium.esp==  LIBRARY+1================14  Enhanced Economy.esp  [Version 3.4.3]15  ProgressArmorer.esp  [Version 1.0]16  ProgressGSD.esp  [Version 2.0]17  ProgressMBSP.esp  [Version 2.0]18  ProgressMercantile.esp  [Version 1.11]19  ProgressRacial.esp1A  ProgressRBSP.esp  [Version 1.0]1B  ProgressSBSP.esp  [Version 1.0]==  CLOTHING=================1C  0FsixyLingerie.esp1D  Colourwheels sixy Lingerie HGEC.esp1E  Crimson Eyecandy Black.esp1F  DM-Lingerie.esp20  ExnemEC_Warrior01.esp21  ExnemWarriorBikini.esp22  Oonas_Outfitters.esp==  TWEAK====================23  [GFX]_Initial_Glow-all.esp24  DrinksForSale.esp25  GP_Lockpicks.esp  [Version 1.5]26  Legion Horses.esp27  P1DanywhereSittable.esp28  P1DrobesOver.esp29  phinix-waterfix.esp2A  ReneersGoldMod.esp2B  Reznod_Mannequin.esp2C  seorin's Artifact Fix.esp2D  Shadow's Horse Wander Fix v1.2.esp2E  Syc_AtHomeAlchemy.esp==  TWEAK+1==================2F  nVidia Black Screen Fix.esp  [Version 1.0.1]30  Oblivion Citadel Door Fix.esp==  TWEAK+2==================31  Hov's Hotkeys.esp32  Kyoma's Journal Mod.esp  [Version 3.0.0]==  OVERHAUL=================33  OOO 1.32-Cobl.esp  [Version 1.72]34  OOO-Respawn_Week.esp  [Version 1.33]35  OOO-Water_Weeds.esp  [Version 1.33]36  Oscuro's_Oblivion_Overhaul.esp  [Version 1.34Beta]==  OVERHAUL+1===============37  Mart's Monster Mod - City Defences.esp  [Version 3.5.5b4]38  Mart's Monster Mod - Diverse Creature Skins.esp  [Version 3.5.5b4]39  Mart's Monster Mod - Foxes.esp  [Version 3.5.5b4]3A  Mart's Monster Mod - Friendlier Factions OOO+FCOM.esp3B  Mart's Monster Mod - Gems & Gem Dust.esp  [Version 3.5.5b4]3C  Mart's Monster Mod - Hunting & Crafting.esp  [Version 3.5.5b3]3D  Mart's Monster Mod - Looting NPCs & Creatures.esp  [Version 3.5.5b4]3E  Mart's Monster Mod - No Blood.esp  [Version 3.5.5b4]3F  Mart's Monster Mod - No Giants.esp  [Version 3.5.5b4]40  Mart's Monster Mod - No Slimes.esp  [Version 3.5.5b4]41  Mart's Monster Mod - Spawn Rates - Reduced.esp  [Version 3.5.5b4]42  Mart's Monster Mod for OOO.esp  [Version 0.9.8b5]==  OVERHAUL+2===============43  LC-Compilation-Exnem.esp44  PAF_armor_additions.esp==  OVERHAUL+3===============45  Kvatch Rebuilt.esp46  Kvatch Rebuilt - OOO Compatibility.esp==  MISC.====================47  Cobl Glue.esp  [Version 1.72]++  Cobl Tweaks.esp  [Version 1.44]48  FFVV_OOO_Birthsigns.esp49  VVAdvanced.esp  [Version 1.00]==  MISC.+1==================4A  BrumaMGRestored.esp4B  DecoratorAssistant with OBSE v1.1.esp4C  Grandmaster of Alchemy.esp4D  Harvest [Flora].esp  [Version 3.0.0]4E  LE_GrowPlants.esp4F  Packhorses v1.esp50  Qarls_Harvest.esp51  Qarls_Harvest addon_MMM.esp52  RealisticFlora.esp  [Version 2.7.0]53  Salmo the Baker, Cobl.esp  [Version 3.08]==  MAGIC====================54  SM_COBL.esp  [Version 0.86]----> Delinquent MASTER: SupremeMagicka.esp55  SM_MMM.esp  [Version 0.89]----> Delinquent MASTER: SupremeMagicka.esp56  SM_OOO.esp  [Version 0.89]----> Delinquent MASTER: SupremeMagicka.esp57  SupremeMagicka.esp  [Version 0.89]==  MAGIC+1==================58  Alchemical Formulas.esp59  Essence Extracter.esp5A  Poison Combiner.esp5B  Poison Maker v1.08.esp==  NPC======================5C  1em_Vilja.esp5D  Aliyah.esp5E  CM Partners.esp5F  CM Partners More NPCs NE.esp60  CM Partners NPC NE.esp61  Rhianna.esp62  Tamara.esp63  TamrielTravellerAdvScript.esp  [Version 1.36]64  TamrielTravellers4MMM.esp  [Version 1.36]65  TamrielTravellersItemsCobl.esp  [Version 1.36]66  TamrielTravellersItemsNPC.esp  [Version 1.36]++  TamrielTravellersItemsVendor.esp  [Version 1.36]==  HOME=====================67  Divine Watch.esp68  Pitchable_Tent_anywhere.esp69  PrincessImperialCityApartment.esp==  QUEST====================6A  300_White Stallion 4.esp==  QUEST+1==================6B  Big Business.esp6C  Big Business Estate.esp==  QUEST+2==================6D  The Ayleid Steps.esp  [Version 3.0.3]==  QUEST+3==================6E  Valley_View_Estate.esp==  QUEST+4==================6F  Windfall.esp==  NONE=====================70  RenGuardOverhaul.esp71  personality_idles2.esp72  Bashed Patch, 0.esp

User avatar
Killer McCracken
 
Posts: 3456
Joined: Wed Feb 14, 2007 9:57 pm

Post » Sun Nov 28, 2010 9:38 am

i strongly recommend:
http://www.tesnexus.com/downloads/file.php?id=24606
:)
User avatar
Chloe Botham
 
Posts: 3537
Joined: Wed Aug 30, 2006 12:11 am

Post » Sun Nov 28, 2010 12:42 pm

Send the .ess and .obse files for one of the affected saves to the team email address in obse's readme. I can look in the co-save to determine if there's any string/array bloat and where it's coming from.

To clarify: the filesize began growing after swapping Living Economy for Enhanced Economy?
EE has been known to leak string variables, though newer versions may address the issue.
You may wish to disable that mod, reload your savegame, resave it, and compare the size of the new .obse file with that of the old.
User avatar
Jay Baby
 
Posts: 3369
Joined: Sat Sep 15, 2007 12:43 pm

Post » Sun Nov 28, 2010 12:10 pm

To clarify: the filesize began growing after swapping Living Economy for Enhanced Economy?
EE has been known to leak string variables, though newer versions may address the issue.
I think I found and removed all leaking string variables in version 3.4.1, so that should no longer be a source of inflating obse co-save file.

But if pre-3.4.1 versions of EE has been in use, clean-saving without EE as scruggsywuggsy suggests, is the way to get rid of the garbage in the save.
User avatar
Rachel Hall
 
Posts: 3396
Joined: Thu Jun 22, 2006 3:41 pm

Post » Sun Nov 28, 2010 7:24 am

I think I found and removed all leaking string variables in version 3.4.1, so that should no longer be a source of inflating obse co-save file.

But if pre-3.4.1 versions of EE has been in use, clean-saving without EE as scruggsywuggsy suggests, is the way to get rid of the garbage in the save.



Thanks for your answers.

I don't think this is specifically related to EE, but if earlier versions where leaky.

I have a .pluggy file from earlier this month 3rd size 11Mb (previous version of pluggy was 125 when this was saved), this is approx. the date I did the switch, so the file was quite large then (but the load/save time wasn't as bad as it is now). Yes I defrag, just in case someome wanted to suggest that.

That gives me 4Mb growth that could be attributed to using 3.3.4 for a while, where did the other 11Mb come from?

My character has been around for well over 700hrs and done most of the big quests and I was hoping to explore Aylied Steps, but it making a coffee whilst saving could be a deterant.

I have checked for bloat with Wrye Bash which shows 22 NullRef objects so haven't removed them, erring on safety. Abomb is at 42% and I have removed that before. But these things are in the Save file which seems to be responding to me" tidying up" in game.

Will try a clean save, and I'll give http://www.tesnexus.com/downloads/file.php?id=24606 ago as well. Anything is better than nothing.

Scruggsywuggsy do you want the pre attempt to clean up or the post?
User avatar
Gen Daley
 
Posts: 3315
Joined: Sat Jul 08, 2006 3:36 pm

Post » Sun Nov 28, 2010 6:15 am

Thanks for your answers.

I don't think this is specifically related to EE, but if earlier versions where leaky.

I have a .pluggy file from earlier this month 3rd size 11Mb (previous version of pluggy was 125 when this was saved), this is approx. the date I did the switch, so the file was quite large then (but the load/save time wasn't as bad as it is now). Yes I defrag, just in case someome wanted to suggest that.

That gives me 4Mb growth that could be attributed to using 3.3.4 for a while, where did the other 11Mb come from?

My character has been around for well over 700hrs and done most of the big quests and I was hoping to explore Aylied Steps, but it making a coffee whilst saving could be a deterant.

I have checked for bloat with Wrye Bash which shows 22 NullRef objects so haven't removed them, erring on safety. Abomb is at 42% and I have removed that before. But these things are in the Save file which seems to be responding to me" tidying up" in game.

Will try a clean save, and I'll give http://www.tesnexus.com/downloads/file.php?id=24606 ago as well. Anything is better than nothing.

Scruggsywuggsy do you want the pre attempt to clean up or the post?

I'd like to see the .obse with the bloat intact. If you want to also send the one that's generated after your attempts at cleaning up that's fine too. But mainly I'm interested in the bloated co-save.
It's important that I have the mod list with which the save was made - so if it doesn't match the one you posted exactly then please include the corresponding .ess along with the .obse.
Thanks.
User avatar
Ezekiel Macallister
 
Posts: 3493
Joined: Fri Jun 22, 2007 12:08 pm

Post » Sun Nov 28, 2010 8:12 am

I think I found and removed all leaking string variables in version 3.4.1, so that should no longer be a source of inflating obse co-save file.

But if pre-3.4.1 versions of EE has been in use, clean-saving without EE as scruggsywuggsy suggests, is the way to get rid of the garbage in the save.



The clean save reduced the .obse file by 15k and increased the .ess by 11K. And as you can tell by the post times two load and two saves takes a while.

So it appears EE is not at fault.
User avatar
Stacy Hope
 
Posts: 3391
Joined: Thu Jun 22, 2006 6:23 am

Post » Sun Nov 28, 2010 9:03 pm

How big are savegames supposed to be?
User avatar
Poetic Vice
 
Posts: 3440
Joined: Wed Oct 31, 2007 8:19 pm

Post » Sun Nov 28, 2010 10:57 am

How big are savegames supposed to be?

:laugh: How long's a pice of string?
Vanilla? Lightly Modded? Types of mods used? Game time played etc?
All thoses variations will give you a different answer to that question.
User avatar
Emerald Dreams
 
Posts: 3376
Joined: Sun Jan 07, 2007 2:52 pm

Post » Sun Nov 28, 2010 7:02 pm

How big are savegames supposed to be?

The rule of thumb is to keep them under 10 MBs. I don't think the co-save size is ever an issue. There were some reports with large .pluggy files and slow load times around v126, but the newer versions use .obse so :shrug:
User avatar
Jenna Fields
 
Posts: 3396
Joined: Mon Dec 11, 2006 11:36 am

Post » Sun Nov 28, 2010 11:35 am

Holy cow! I am running into co-save issues using OBSE 18 and Pluggy 125. The co-save is 33MB! I disabled the mod and saved and the co-save went to 456 bytes. Now, the person to blame is me as I am developing the mod, but I am hoping to determine why it is so big. It seems to be related to data I pull from a file and somehow it is not releasing arrays or something. I use arrays to store the data I pull from the pluggy file. It is supposed to overwrite what was there when the game loads, but it is not. It is referencing something and bloating the obse file.

Couple of questions:
1. Will updating Pluggy fix this?
2. What do I need to do to make sure the arrays are getting de-referenced and deleted?
3. Is there a better way to read an INI style file than by using Pluggy? (Maybe I will dump Pluggy and use esp files. I like Pluggy as it is easily user extendable for settings.)
4. If I dereference an array that contains arrays will it delete the sub arrays?
5. Is this a Pluggy issue with not deleting something?

Info:
OBSE 18
Pluggy 125b
Converting pluggy strings to OBSE strings via abusing an item by setting its name to the pluggy string then reading back using OBSE function.
Relying on OBSE to detect deference of arrays for deletion. May be using arrays in arrays too. In fact, I am sure I am. Explicit de-reference of arrays?
User avatar
ONLY ME!!!!
 
Posts: 3479
Joined: Tue Aug 28, 2007 12:16 pm

Post » Sun Nov 28, 2010 12:58 pm

Holy cow! I am running into co-save issues using OBSE 18 and Pluggy 125. The co-save is 33MB! I disabled the mod and saved and the co-save went to 456 bytes. Now, the person to blame is me as I am developing the mod, but I am hoping to determine why it is so big. It seems to be related to data I pull from a file and somehow it is not releasing arrays or something. I use arrays to store the data I pull from the pluggy file. It is supposed to overwrite what was there when the game loads, but it is not. It is referencing something and bloating the obse file.

Couple of questions:
1. Will updating Pluggy fix this?
Quite surely not. It is most likely some temporary strings in your scripts that never get released.

2. What do I need to do to make sure the arrays are getting de-referenced and deleted?

Learn their ways ;)

Wrye Bash has a command named something like ".obse statistics" which you can call by right-clicking on a save. It will open up a popup with all data in the .obse co-save file. To weed out bloat in my mod, I opened such a co-save, pasted it into a text editor. Then did something I suspected increased bloat, saved, opened the co-save for the new save, and compared the two. Not a simple job, but it helped me.

3. Is there a better way to read an INI style file than by using Pluggy? (Maybe I will dump Pluggy and use esp files. I like Pluggy as it is easily user extendable for settings.)

For lots of strings, not as far as I know. But for settings, OBSE's RunBatchScript is the simplest and best.

4. If I dereference an array that contains arrays will it delete the sub arrays?
Yes

5. Is this a Pluggy issue with not deleting something?
In my use of Pluggy strings, I only use the temporary strings (with numbers as IDs) to avoid any such issue, so I don't know how much problem it can be.
Info:
Converting pluggy strings to OBSE strings via abusing an item by setting its name to the pluggy string then reading back using OBSE function.
Upgrade Pluggy and use the ToOBSE function instead.

Relying on OBSE to detect deference of arrays for deletion. May be using arrays in arrays too. In fact, I am sure I am. Explicit de-reference of arrays?
The only place I found it necessary to explicit de-reference OBSE arrays, was if an array (t_ar) already pointed to something before being used on the left side in a ForEach t_ar <- m_ar. Strings defined in user functions MUST be explicitely de-referenced before the function script returns though, which was the major cause of bloat in my mod.
User avatar
JLG
 
Posts: 3364
Joined: Fri Oct 19, 2007 7:42 pm

Post » Sun Nov 28, 2010 10:38 pm

The rule of thumb is to keep them under 10 MBs. I don't think the co-save size is ever an issue. There were some reports with large .pluggy files and slow load times around v126, but the newer versions use .obse so :shrug:

Any known issue with them being over 10 mbs? Mine were 15 mb but then I removed 12 mods for good and it reduced the size of the save games to 14 mb.

So save game size is related to mod amount, but I'd not heard that there was a 10 mb rule of thumb kind of thing.
User avatar
LuCY sCoTT
 
Posts: 3410
Joined: Sun Feb 04, 2007 8:29 am

Post » Sun Nov 28, 2010 3:04 pm

TheNiceOne,
You gave me a lot to think about. What I did was monitor file size for a clean save and could see where the file was increasing in size. Then I went ahead and isolated the routine I suspected was increasing the bloat. The bloat went away. Then I concentrated on the pluggy stuff because the obse arrays of data I was building were not getting bigger: ar_dump showed normal size. Then I figured it was a variable not being deleted from pluggy: this is the only place I use pluggy is one routine.

This is how I fixed it:
IniReadString rStr file section key dStr  ; Read string from iniset attribval to Call WOLToOBSEString rStr  ; Converts from pluggy string to obse string using a hack (v125b of pluggy).  This works well.let rStr := Call WOLRecyclePluggyString rStr  ; This function explicitly deletes the pluggy string and creates a new one for use.


Here is how WOLRecyclePluggyString is defined:
scn WOLRecyclePluggyStringlong pluggyStringbegin Function{pluggyString}	  DestroyString pluggyString  set pluggyString to CreateString -1 "%e"	  SetFunctionValue pluggyStringend


So, now my save file stays the same size each time. I also did some things to reduce OBSE array bloat. I have a temporary array that reads in the NPCs that are nearby to track one shot kills and other things. Whenever I open a dialog I reset the size of that array to 0. This should de-reference everything in the array and keep the save game dialog from saving that junk. It is not needed and is rebuilt as soon as a menu is left. Anyway, I will update this if I find more bloat areas, but it seems to be my improper use of pluggy strings assuming they would self clean to some degree. They don't and you must delete the string and create a new one each time.
User avatar
Susan Elizabeth
 
Posts: 3420
Joined: Sat Oct 21, 2006 4:35 pm

Post » Sun Nov 28, 2010 11:07 pm

Well, save files are supposed to grow. Slowly as you said.
The longer you played with this character, themore things you've done, the more items, houses and horses you possess,the more quest and companions, the bigger the save.
At 90 hours of game, 62 levels, over 70 armor&weapons adding modsmy save was 13 MB. When i read your topic and cleaned it, the save is 11.3 MB. Same thing. No big difference.
I consider this normal with almost 400 esp+esm files...

if you want to stay under 1 MB, don't leave the sewers.
User avatar
Nomee
 
Posts: 3382
Joined: Thu May 24, 2007 5:18 pm

Post » Sun Nov 28, 2010 9:12 pm

Well, save files are supposed to grow. Slowly as you said.
The longer you played with this character, themore things you've done, the more items, houses and horses you possess,the more quest and companions, the bigger the save.
At 90 hours of game, 62 levels, over 70 armor&weapons adding modsmy save was 13 MB. When i read your topic and cleaned it, the save is 11.3 MB. Same thing. No big difference.
I consider this normal with almost 400 esp+esm files...

if you want to stay under 1 MB, don't leave the sewers.

Who are you responding to?
User avatar
Sarah Kim
 
Posts: 3407
Joined: Tue Aug 29, 2006 2:24 pm

Post » Mon Nov 29, 2010 1:23 am

Still have some serious bloat going on somewhere else. Enough that I am rewriting the whole mod. There are a lot of things I did in the mod early on I would not do now. So this will be a good way to modularize the code even more and leave out the junk I don't need. So far I have been testing piece by piece that deal with storing any data and monitoring for bloat. This has made me very wary of how often I use temporary strings especially in arrays that I deref. There were enough entry points to the code, and enough "not sure how the program flows" questions that I knew I needed to clean it out. Rewriting is the way to go.
User avatar
helen buchan
 
Posts: 3464
Joined: Wed Sep 13, 2006 7:17 am

Post » Sun Nov 28, 2010 10:22 am

I've developed a habit that I believe keep save game size down. Whenever dungeon diving, I don't drop items I don't want, but put them back on dead enemies, or in containers. That way, the unwanted stuff gets deleted when the cell resets. Things just dropped on the ground are retained in the save.

To start with, I did it so it was easier to pick stuff out to haul back to town for sale. It's much easier to look through a sorted corpse inventory than a pile on the ground. But then I saw the effect on the save size, and kept doing it for that reason.
User avatar
Haley Merkley
 
Posts: 3356
Joined: Sat Jan 13, 2007 12:53 pm

Post » Mon Nov 29, 2010 1:25 am

Any known issue with them being over 10 mbs? Mine were 15 mb but then I removed 12 mods for good and it reduced the size of the save games to 14 mb.

So save game size is related to mod amount, but I'd not heard that there was a 10 mb rule of thumb kind of thing.

As save file size increases, load times increase significantly, and as you get closer to about 15 MB, performance deteriorates rapidly. By 20 MB, the game is seeing all kinds of weird glitches as things stop fitting together correctly. By 30 MB, the save file is probably unplayable. (These figures are all very, very generous. For most people on most systems, a save file even approaching 20 MB will be completely unplayable, and performance will be noticeably affected by 10 MB)

dev_akm did a ton of testing on this way back when, so he would have been the go-to guy for those questions. But yes, after 10 MB, I'd be worried about a save file - you really don't want it getting much past that.

This is for .ess files only. As far as I know, there are no issues with large .obse co-save files aside from wasting disk space.
User avatar
My blood
 
Posts: 3455
Joined: Fri Jun 16, 2006 8:09 am

Post » Sun Nov 28, 2010 11:48 am

I've developed a habit that I believe keep save game size down. Whenever dungeon diving, I don't drop items I don't want, but put them back on dead enemies, or in containers. That way, the unwanted stuff gets deleted when the cell resets. Things just dropped on the ground are retained in the save.

To start with, I did it so it was easier to pick stuff out to haul back to town for sale. It's much easier to look through a sorted corpse inventory than a pile on the ground. But then I saw the effect on the save size, and kept doing it for that reason.


Clean Up does that automatically but instead of putting the items back on enemies, it disables them. Much faster.

@ DragoonWraith,

Was his testing done on a modded game or vanilla?
User avatar
Matthew Barrows
 
Posts: 3388
Joined: Thu Jun 28, 2007 11:24 pm

Post » Sun Nov 28, 2010 6:54 pm

For my FCOM setup (total 254 active mods, plus many others merged in bashed patch), my save game size is about 13MB at 140 hours of playtime. I use kuertee Cleanup, and have no bloating. Is this normal? What are your save game sizes?
User avatar
Max Van Morrison
 
Posts: 3503
Joined: Sat Jul 07, 2007 4:48 pm


Return to IV - Oblivion