Mod request/discussion - revised theft

Post » Sat May 28, 2011 6:28 pm

I am catching up on reading the documentation for some of the modding tools. I'm more than a bit of a programmer, so I'm finding the possibilities fascinating to tweak mods I nearly like, and to clean up and explore others for tips if I venture down that route myself, for personal fun or wider circulation...

In the Enchanted Editor manual, there's this fascinating section (my highlighting):

Reduce a 5MB Save Game File to 1MB (FPS increase and kills most "doubling"")

By Farren

Originally redistributed by Argent (thanks!)

This technique has some quirks like resurrecting all dead characters but does not break quests or change characters attitudes to you etc. The vast majority of space taken in in your save is NPCC (characters you've stolen from, killed etc) and CELL (Cells you've altered) records, so...

1. Load the save game

2. Turn OFF safe editing (Edit/Editing Options/Safe Editing or just click the symbol like a roadsign on the toolbar till it warns you and the symbol turns bright yellow and red)

3. Click the check box next to the "Cells" node to mark all cells.

4. Click the "Cells" node (not the checkbox) to expand it and show all cells and scroll down till you see the cell or cells where you keep all your loot (your home, if you have one). Uncheck this cell in the check box or you lose your loot.

5. Click the check box next to "NPCs Altered/Killed by Player" (NPCC) node to mark all altered NPCs.

6. CRITICAL: Click the "NPCs Altered/Killed by Player" (NPCC) Node to expand the node and show all NPCC record. Scroll down to the "PlayerSaveGame" record and UNCHECK it. You don't want to delete this specific record or your save game will be ruined.

7. Press Delete. All CELL and NPCC entries apart from your home and the PlayerSaveGame will be deleted.

8. Save your game. I got a 5MB game (LOTS of mods and Neverine) down to 1MB like this. Everyone I knew still thought I was the bees knees and I could walk around Balmora without feeling like a crippled snail on downers afterwards (much better FPS).


This made me think that there's the possibility for a VERY useful mod, that would have wider benefits than just the gameplay aspect.

The basic idea is this: after a certain amount of time has elapsed (2 weeks of game time?) have the NPC traders "forget" that you've stolen stuff from them. The rationale for the stealing code is that you shouldn't be able to steal and then sell stuff straight back to the vendor, but having such a long window would prevent that, since all but the most bloody-minded player would just find an alternate vendor to fence their goods to in the meantime, so there would be almost no practical difference in gameplay other than making traders "usable" again after a cooldown time. Given the newly restrictive changes on gameplay (enchanted item cooldown etc.) being forced on us by the latest MCP, this isn't such a bad rebalance. ;)

It also avoids to a large extent the unreal nonsense that I sell a vendor a chunk of (say) muck I've just collected myself and they identify it as the EXACT same one I stole some game months ago when I was just starting out and testing out my sneak skills. :( It's a bad kludge. Using the console to check ownership of every minor item I want to sell is NOT immersive, nor is having to save before every trade, if you happen to be playing a thief character.

The major side benefit I alluded to above is that removing the list of stolen items from the trader after a set length of time will remove that list from the save games, getting the slim-down effect reported in the Enchanted Editor manual. In checking ownership in the console, the list doesn't just include traders, apparently. AFAICT anything you steal from an NPC (or from a chest owned by them??) is flagged and stays in the list, even if you kill that NPC or you wouldn't be able to sell it back to them anyway. How do I sell stuff back to "Great House Hlaalu" - there's no NPC by that name? So this list only gets longer and longer over time, bloating the save games with much totally unneeded information.

As I'm not a modder (yet!) I don't really know how feasible this is to implement. Off the top of my head, from what I've understood of the game mechanics so far, I wonder if a dummy item could be added to the "stolen from NPC 'X'" list that encodes the game date of the event somehow, or a global variable is set. Then a script somewhere that runs occasionally (the sleep script?) could check for all outdated references prior to the cutoff time and remove them from the game, effectively making the NPCs forget the theft and cleaning all future save games. The sleep script would be ideal since by default it involves a game autosave anyway, so cleaning just before that takes place would be perfect. I guess this might be something the MCP would be better at implementing though, since it likely involves reaching parts of the game mechanics normal scripting cannot reach. ;)

If anyone is interested in brainstorming and possibly implementing this, I'd be very interested to see it. And I suspect the impact for everyone in terms of save game slimming would be a major result.
User avatar
Cassie Boyle
 
Posts: 3468
Joined: Sun Nov 05, 2006 9:33 am

Post » Sat May 28, 2011 5:15 pm

I don't think something like this could be implemented in-game. Perhaps with the MCP, but this isn't the type of thing that Hrnchamd seems to be focusing on for the patch.
And for many instances there would be times when you would not want to erase these references... even if it's just for Roleplaying.

When you edit your Save file manually, you can remove or keep exactly what you want. I think a lot of us do this already, if we have a huge Save file.

(I always erase a lot of the random Containers and Creatures Killed... because there are so many references, and erasing them is safe.)



I almost always play Stealth characters, and I very rarely encounter any problems with Ownership and/or selling items.
Just never steal anything from your favourite merchants and you'll be safe. Seems pretty straight-forward to me.
User avatar
Nick Pryce
 
Posts: 3386
Joined: Sat Jul 14, 2007 8:36 pm

Post » Sat May 28, 2011 7:33 am

And, not to shoot down another good idea; but we don't have any control over what is written to the save game file in-game, so you might have to consider a utility for outside of game to do this. Even with MWSE, which might be able to do this (alter a save file) the problem is that then the MW engine will re-write the save file all over again to its original contents at next save.

A discussion about changing the character's hair/facial features a little while back talked about these issues. Post is http://www.gamesas.com/index.php?/topic/1160703-life

The only way I forsee something like this is one of two ways:
a new MW engine (like "openMW") and if the devs add a feature that allows changing specifics in a save file -OR- an interrupt to the write to save from the engine... and that is way beyond my meager programming skills

ST
User avatar
Anna Watts
 
Posts: 3476
Joined: Sat Jun 17, 2006 8:31 pm

Post » Sat May 28, 2011 11:44 am

When you edit your Save file manually, you can remove or keep exactly what you want. I think a lot of us do this already, if we have a huge Save file.


I wonder how many players do that, actually, who aren't accomplished modders. If I hadn't ended up here I wouldn't have dreamed of editing a save game, or even have known it was possible through the EE. My major point is that trimming the save games automatically would benefit everyone, and there's a valid gameplay aspect to it that can be argued just as strongly as the other personal gameplay "fixes" that aren't bugs which hrnchamd is forcing through the MCP...

I'd also point out that the MCP already hooks into the save game code - hasn't it implemented the save-cleaning system from Wrye Mash? So there's already precedent for "cleaning" saves and this might not be too difficult to add within that without scripting or anything else. All it needs is the dummy date item adding to the list of stolen items for an NPC (which wouldn't affect any gameplay since the engine only compares item IDs I guess), and when those particular records for the NPC are written out in the save game, the whole subrecord set can be skipped if the date is past the threshold. Doing it with a persistent date script-type variable (global? local?) stored for the NPC would be even easier, which would be dumped/reset when the cleanup was triggered. If that would work then the stolen list would trim correctly on every save.

From the ESTemplate.ini that's part of the Enchanted Editor manual, I guess we'd be dealing with one of these entries, probably the first:

NPCC.NPCO.ItemID = BOOK,MISC,LIGH,WEAP,ARMO,LEVI,CLOT,INGR,ALCH,APPACNTC.NPCO.ObjectID = BOOK,MISC,LIGH,WEAP,ARMO,LEVI,CLOT,INGR,ALCH,APPA


OK, here's the concept in pseudocode:

Either through scripting or a code patch, add the function that when the PC successfully steals an item, the owner NPC has an integer variable (somewhere) updated with the current game day. Is there a spare 2-4 bytes somewhere in the NPC data this could be slipped in? Since the game starts at day 1, an unset value of 0 also makes this a simple Boolean flag (true/not true) for stolen from/not stolen from. That's it.

On a save, on iterating through the data chunks, when an NPC node is reached, get this variable from wherever it is stored in the datastream. let's call it stolenday.

checkday = currentday - threshold_days    <--- can be done once at save start; loop getting and saving datachunks for this NPC until the chunk with the encoded stolenday variable is reachedif ( stolenday AND stolenday <= checkday )    stolenday = 0    reset this variable to 0 in the datachunk we just got it from, and write it out as part of the new saveendif; now, while continuing to loop getting data chunks for this NPCif ( chunkheader == NPCC AND stolenday==0)    skip writing out this chunk as the NPC's stolen list is being fixed/updated


That's it. It's trivial code to add - I've done x86 ASM programming so I'm not talking out of my a$$. For someone with hrnchamd's knowledge of the MW EXE it would be simple. All it then needs to make old savegames compatible with adding this mod is minor sanity checking on a normal file load to set any NPCs wth NPCC chunks (have had items stolen from them) that don't have an initialised stolenday variable get it set to the savegame's game day, otherwise initialised to 0.

Without additional scripting a pure patch version as outlined above would only update on save/load. So if (for instance) you played a game for two game weeks solid, no save, any traders you'd stolen from on day 1 wouldn't update on the 15th day - behaviour would be as the current game. How likely is a two week unsaved gamespan tho? As soon as you'd saved and loaded a file all the traders would be updated.

BTW, in the manual for Enchanted Editor, there's a link to a full technical description of the MW file structure by Dave Humphrey, which was at http://members.optusnet.com.au/argent2/ESM_file_format.htm - this link is now dead. Does anyone have a current link to that information?
User avatar
Zach Hunter
 
Posts: 3444
Joined: Wed Aug 08, 2007 3:26 pm

Post » Sat May 28, 2011 10:33 am

Well, I will say this: there are plenty of things we knew "couldn't be done" in the past... such as compiling scripts into the game from an outside program (MSWE) or catching the calls to the direct x graphics handlers (MGE). Imagine what else the future will show that the statement "can't be done" is wrong about. So by all means, give it a go, and let us know how it goes. If you could figure out a way to modify what the engine writes to a save file you would be hailed as a hero. My recommendation would be to talk to Hrnchamd cuz nobody knows more about the engine and the processes inside.

As for the file for the full technical description of the MW file structure, I can give it to you... but it is not as full and complete as you might wish. It still is useful tho.

ST
User avatar
Olga Xx
 
Posts: 3437
Joined: Tue Jul 11, 2006 8:31 pm

Post » Sat May 28, 2011 9:44 am

Pseudocode logic in post 4 fixed. ;)

Yep, this is only possible if Hrnchamd adds it into the patch. It rather depends on how the patched save routine handles the datachunks. If it parses each one individually as I've suggested in the pseudocode then this should be a trivial fix; if it grabs whole NPC records and saves them out as a bundle then it would take much more code to parse through them. What I've outlined above also doesn't handle a situation where a threshold is passed during an active game and you visit a trader who should now be reset to an unstolen-from state. Only by saving and loading the game would that be updated. But since that's the same as the current game situation and most people probably save often, that's less of an issue.

The interesting thing is it opens the possibility to do other sorts of "cleaning" on save games as Pluto suggested. I don't think I'd want to play a game where I revisited a cell only to find characters and unique monsters have magically resurrected, or loot containers have reset all over the place. But there are likely other persistent references in the games that could be stripped out at certain points to slim down the saves. Hooking into the save code this way would allow such automatic cleanup.
User avatar
Nienna garcia
 
Posts: 3407
Joined: Wed Apr 25, 2007 3:23 am

Post » Sat May 28, 2011 6:39 am

I guess, if I steal an item one day, and then steal a different item the next day the NPC would get an updated stolenday variable. So all items from that npc will clear at the same time I guess.

I've actually thought about this myself many times. I did come up with something for containers too.

After If altered container is empty, check if owner is alive. If owner is alive, clear atlered-flag on container.
User avatar
Taylrea Teodor
 
Posts: 3378
Joined: Sat Nov 18, 2006 12:20 am

Post » Sat May 28, 2011 7:04 am

I guess, if I steal an item one day, and then steal a different item the next day the NPC would get an updated stolenday variable. So all items from that npc will clear at the same time I guess.

Yes, but if you're continually stealing from the same merchant then there has to be some result. Perhaps if they are having a long run of thefts then for a while they are going to be less trusting of people selling them stuff. ;)

I've actually thought about this myself many times. I did come up with something for containers too.

After If altered container is empty, check if owner is alive. If owner is alive, clear atlered-flag on container.

That's exactly the sort of additional auto save-cleaning I'm talking about, removing unnecessary persistent ownership information when the owner is dead. But doesn't the altered flag prevent the container respawning its original specified contents when you revisit it, rather than keeping what you've left in it, if anything? I'm not sure that's what you want...
User avatar
JLG
 
Posts: 3364
Joined: Fri Oct 19, 2007 7:42 pm

Post » Sat May 28, 2011 8:22 am

That's exactly the sort of additional auto save-cleaning I'm talking about, removing unnecessary persistent ownership information when the owner is dead. But doesn't the altered flag prevent the container respawning its original specified contents when you revisit it, rather than keeping what you've left in it, if anything? I'm not sure that's what you want...


Actually, containers are filled with leveled lists. So it will not be the original contents. The contents will be randomly selected from the leveled list(s) again. As for 'keeping what you've left in it" thats why I added the 'if the container is empty' check. That way only empty containers get cleared. Containers that still have items in them should remain unchanged.
User avatar
Britney Lopez
 
Posts: 3469
Joined: Fri Feb 09, 2007 5:22 pm

Post » Sat May 28, 2011 4:16 pm

Hmm, there's a tweak to this idea that would improve gameplay even further. As the proposal stands, the "forgetfulness" of the trader would only happen if you successfully stole from them without them noticing. What could easily be added is an extra setting for the "stolenday" variable of -1, in the case that you stole and were caught. In that case, the trader would never forget and any items you try to barter that were currently marked as stolen (and that you might steal in future) from them would always be reported, just as the current game works, on the (let's face it, entirely correct) assumption that you were the original thief. That would rebalance this in favour of the trader again.
User avatar
CHANONE
 
Posts: 3377
Joined: Fri Mar 30, 2007 10:04 am


Return to III - Morrowind