[Important] Concerns regarding deleted references

Post » Sat May 26, 2012 7:43 am

As Skyrim has seen a healthy influx of new modding talent, I feel that it is very important to warn the growing community about a very easy trap to fall into that can cause your mods to destabilize the game: deleted references.

When a mod deletes a reference that does not belong to it, the game has a tendency to crash on exit (because it is trying to unload objects that no longer exist). But that is not the worst of it. If your mod deletes a reference, and then ANOTHER mod comes along and edits that reference, this will cause the game to crash when it tries to load that reference. So, say mod A deletes a rock in Whiterun to make room for a new building, but then mod B comes along and moves that rock somewhere else for its own purposes. If you use mod A and mod B together, your game will crash when you enter Whiterun because mod B is trying to edit an object mod A deleted.

Finally, in Skyrim, if you delete an object in the Creation Kit, there is a chance that it won't actually be removed, but instead moved to the 0,0 position of whatever cell or worldspace they occupy. This bug is new, while the previous two potential crashes have been with us since Morrowind.

So remember, NEVER delete objects that don't belong to your mod if you can help it. Instead, flag the reference you want gone as "Initially Disabled" and, if you want to be very sure, set its Z position data to -30000. But NEVER delete it, or you open the floodgates for all sorts of problems. ONLY delete references that you are certain belong to your mod.
User avatar
NO suckers In Here
 
Posts: 3449
Joined: Thu Jul 13, 2006 2:05 am

Post » Sat May 26, 2012 8:50 am

To late Maestro. The Workshop and S Nexus is full with Mods, that does exact this an other no goes :D. I think that it is a good Idea to Pin a Thread on the Top of this Forum here with a List of =>"Dont do this" ....
User avatar
Tarka
 
Posts: 3430
Joined: Sun Jun 10, 2007 9:22 pm

Post » Sat May 26, 2012 11:10 am

To late Maestro. The Workshop and S Nexus is full with Mods, that does exact this an other no goes :biggrin:. I think that it is a good Idea to Pin a Thread on the Top of this Forum here with a List of =>"Dont do this" ....

That's why I posted this. ;)
User avatar
Loane
 
Posts: 3411
Joined: Wed Apr 04, 2007 6:35 am

Post » Sat May 26, 2012 11:29 pm

Another of the many reasons why I'm not using the CK until it gets patched.
User avatar
lauren cleaves
 
Posts: 3307
Joined: Tue Aug 15, 2006 8:35 am

Post » Sat May 26, 2012 4:22 pm

Does the CK have the ability to disable instead of delete?
User avatar
Sabrina garzotto
 
Posts: 3384
Joined: Fri Dec 29, 2006 4:58 pm

Post » Sat May 26, 2012 11:42 am

Ugh, one more issue that I have to worry about. Both as a player as well as a modder. They really seem to have dropped the ball this time around with the CK. While it does add a lot of cool new features and functions, the bugs are flat out ridiculous. I mean, have you seen this thread? http://www.gamesas.com/topic/1352954-unofficial-creation-kit-modding-bug-list-3/ That list is massive, and the bugs are sooo bad.
User avatar
koumba
 
Posts: 3394
Joined: Thu Mar 22, 2007 8:39 pm

Post » Sat May 26, 2012 3:10 pm

Ugh, one more issue that I have to worry about. Both as a player as well as a modder. They really seem to have dropped the ball this time around with the CK. While it does add a lot of cool new features and functions, the bugs are flat out ridiculous. I mean, have you seen this thread? http://www.gamesas.com/topic/1352954-unofficial-creation-kit-modding-bug-list-3/ That list is massive, and the bugs are sooo bad.
It has been this way since Oblivion, which is why many of us advocated cleaning mods. A recommendation that angered many mod makers.
User avatar
Tracy Byworth
 
Posts: 3403
Joined: Sun Jul 02, 2006 10:09 pm

Post » Sat May 26, 2012 11:29 pm

With all the lessons learned with Oblivion modding, I am surprised we do not have a pinned post with guidelines to creating mods, a standard if you will. When I first played Skyrim I was amazed to not get any CTDs or lag problems. Played 100+ hours, even used some mods (pre-CK mods) and the game was rock solid. Now I get CTDs with some new mods. I understand it is still a discovery process for modders, but I will refuse to go back to the OB CTD days! So BRAVO Thomas Kaira for starting this and BRAVO R3NNW for suggesting this gets pinned (or at least a modders guide). Lets nip this early, and as Psymon suggested, encourage modders to clean their mods if and when this is possible.
User avatar
Britta Gronkowski
 
Posts: 3475
Joined: Mon Apr 09, 2007 3:14 pm

Post » Sat May 26, 2012 6:07 pm

The Creation Kit has options to disable references (an Initially Disabled checkbox in the Reference options menu), but that takes a lot more time because you need to double-click every reference and then select the checkbox. Thus modders who are unaware of the problems it can cause tend to take the "lazy road" and delete everything they don't want. Much faster, and can be done in bulk, but now your mod is subject to the above, and that is a situation no modder wants to find themselves in (knowing your mod is causing people's games to crash).

This will stand until we get a tool that allows us to easily undelete objects (TESVEdit, for instance).

Deleting things added by your own mod is fine, though. Those will get removed from the plugin directly. This only applies to deleting objects that do not belong to your mod (that is to say, everything in the game when you fire up the Creation Kit for the first time).
User avatar
Jonathan Braz
 
Posts: 3459
Joined: Wed Aug 22, 2007 10:29 pm

Post » Sat May 26, 2012 11:42 am

Since it appears this is an issue, and there is no TES5Edit yet, I can expose the UDR functionality in Wrye Bash once I'm done with this Load Order junk. Luckily, scanning for UDR's doesn't require any understanding of the records themselves, since whether it's deleted is all in the flags in the record header. The only bit I'll actually have to program in is which record types shouldn't be deleted (ACHR, REFR in Oblivion, probably something similar in Skyrim). Though maybe someone could pipe in if they happen to know.

I can't actually program in the fixing of the record, since that does require decoding of records (need to load up the last master that has a valid version of that record and restore it from that), but I can at least show which records are deleted and need some cleaning.

Edit: just took a look at the code, and Wrye Bash already has UDR scanning enabled for Skyrim. There's a few cases where it'll throw an error, so I've fixed those, and about to commit them to the SVN. Also enabling UDR scanning for the 'Create BOSS Report...' command as well. So yeah, you can already scan for UDR's with Wrye Bash, but no ITM's (that'll require CBash to support Skyrim, which in turn will require all of the record types decoded first).

Edit 2: Though I'd like to test it out, if anyone knows of a mod that has a deleted reference, post a link here and I'll check the scanning of it.
User avatar
Eoh
 
Posts: 3378
Joined: Sun Mar 18, 2007 6:03 pm

Post » Sat May 26, 2012 8:27 pm

Does the CK have the ability to disable instead of delete?

The Delete Button is only to erase your own new placed Objects. When you want to delete Vanilla Objects, you must activate "Version Control" in your CK, check the Forms from the ESM out, delete them and check the new ones in. But this alters the Master File :D.



Sry for my bad english, but i'm from austria.
User avatar
Harry Leon
 
Posts: 3381
Joined: Tue Jun 12, 2007 3:53 am

Post » Sat May 26, 2012 3:08 pm

The Delete Button is only to erase your own new placed Objects. When you want to delete Vanilla Objects, you must activate "Version Control" in your CK, check the Forms from the ESM out, delete them and check the new ones in. But this alters the Master File :biggrin:.

Well considering that you can't redistribute the skyrim.esm, that doesn't help much does it? Please don't confuse the issue. You can not delete objects from the base game. You have to disable them and move them out of view.

Also this is NOT a bug in the Creation Kit. The object was added and placed in the Skyrim master. Why would you expect that you can delete it with no consequences? You have no way of knowing if that object will be referenced by another mod - and that's when problems occur. So please take some care up front. Once TES5Edit is released, the Undelete and Disable References function will take care of this for you, but until then, you need to do it yourself.
User avatar
James Smart
 
Posts: 3362
Joined: Sun Nov 04, 2007 7:49 pm

Post » Sat May 26, 2012 8:02 am

So, say mod A deletes a rock in Whiterun to make room for a new building, but then mod B comes along and moves that rock somewhere else for its own purposes. If you use mod A and mod B together, your game will crash when you enter Whiterun because mod B is trying to edit an object mod A deleted.
This hasn't been my experience. Yes, I've been a bad boy and released OCS with deleted records in place :P

Also my testing with my conflicting mods pair has shown that the only issue is at exit. Crashing in-game due to a deleted reference hasn't happened yet. Though I haven't got a whole slew of these in the testing pair either.

Another of the many reasons why I'm not using the CK until it gets patched.
Don't expect this to ever get patched btw. As far as the CK, hitting "delete" does exactly what it's supposed to. The game itself is what's not behaving properly.

Does the CK have the ability to disable instead of delete?
It does, check the "initially disabled" box and then set the Z position to -30000. Extremely tedious though.

Edit 2: Though I'd like to test it out, if anyone knows of a mod that has a deleted reference, post a link here and I'll check the scanning of it.
I'll give it a run. I have deletions in OCS I'd rather not have there if I can help it.

What I'm not going to end up doing though is getting all caught up in the BS of using MasterUpdate to solve the game's problems as became commonplace in FO3 modding. If things start going down that road, forget it. Not worth dealing with grossly implemented solutions of that nature.
User avatar
Natalie J Webster
 
Posts: 3488
Joined: Tue Jul 25, 2006 1:35 pm

Post » Sat May 26, 2012 11:12 am

This hasn't been my experience. Yes, I've been a bad boy and released OCS with deleted records in place :tongue:

Also my testing with my conflicting mods pair has shown that the only issue is at exit. Crashing in-game due to a deleted reference hasn't happened yet. Though I haven't got a whole slew of these in the testing pair either.

Of course, volume is a factor. The more NULL pointers, the higher the likelihood of data getting corrupted down the line. We just need to remember that all the old rules from Oblivion concerning deleted refs still stand, and it is still important that they be avoided.

And yes, I hated that MasterUpdate stuff in Fallout 3 something terrible. I have refused to ever use it and my game is as stable as Fallout 3 can possibly be (which is to say, not brilliant) with all my mods. Same deal with New Vegas. Hackish workarounds are, and always will be, hackish.
User avatar
Connie Thomas
 
Posts: 3362
Joined: Sun Nov 19, 2006 9:58 am

Post » Sat May 26, 2012 6:16 pm

Of course. I'm not disputing that, and I am eagerly looking forward to anything we can use as a solution to deal with them in bulk. Mad clicking in the CK to disable stuff isn't how I plan to spend my time :P
User avatar
Gemma Archer
 
Posts: 3492
Joined: Sun Jul 16, 2006 12:02 am

Post » Sat May 26, 2012 10:48 am

Very sorry to necro this thread, but I found it an important topic. Having just come back to Skyrim modding after going back to Oblivion whilst waiting for the mod-community to come "far enough", I decided to move back here. Now, I have a few mods that have massive UDRs (One of them had 122 UDRs for example) but I can't for the life of me find a way of cleaning them. Has any progress been made when it comes to this, or will we simply need to wait for TESVEdit (Granted it will be released at all)? Sorry if I missed another post elsewhere, but every search I tried ended up giving me just links to BOSS/Wrye Bash threads where UDRs were mentioned and no real information. Thanks!
User avatar
leigh stewart
 
Posts: 3415
Joined: Mon Oct 23, 2006 8:59 am

Post » Sat May 26, 2012 12:24 pm

I'm curious how you even detected them?
User avatar
Sheila Esmailka
 
Posts: 3404
Joined: Wed Aug 22, 2007 2:31 am

Post » Sat May 26, 2012 10:08 pm

I'm curious how you even detected them?

WB mod checker function lists the UDRs (not ITMs though), just can't do anything about them for Skyrim.
User avatar
Jeneene Hunte
 
Posts: 3478
Joined: Mon Sep 11, 2006 3:18 pm

Post » Sat May 26, 2012 6:58 pm

Bash SVN can detect them, but not act on them. We're stuck waiting for a useful tool to clean these up with.

No, there's no word yet on when TES5Edit is coming. It can't get here soon enough though.
User avatar
Adrian Morales
 
Posts: 3474
Joined: Fri Aug 10, 2007 3:19 am

Post » Sat May 26, 2012 11:23 am

So remember, NEVER delete objects that don't belong to your mod if you can help it. Instead, flag the reference you want gone as "Initially Disabled" and, if you want to be very sure, set its Z position data to -30000. But NEVER delete it, or you open the floodgates for all sorts of problems.
That won't always work, like if a reference is saved as enabled or with its original coordinates, it'll still render where it always did. Instead, as *Edit does, http://i.imgur.com/Pw8vV.png. The player is persistent and is never disabled, so the parented references will assuredly not render even if saved as enabled.

To remove "deleted" references from a plugin, open it with the CK, then File > Data > Details and sort by "Deleted", select/multiselect any and all "Deleted" cell children noting their FormID's, then hit the delete key to mark them "Ignored" and save. The next time the CK closes, all "Ignored" forms will be removed. From there, one can revisit the forms and remove them via enable parentage as described above.
User avatar
Andrew Perry
 
Posts: 3505
Joined: Sat Jul 07, 2007 5:40 am

Post » Sat May 26, 2012 10:45 pm

So remember, NEVER delete objects that don't belong to your mod if you can help it. Instead, flag the reference you want gone as "Initially Disabled" and, if you want to be very sure, set its Z position data to -30000. But NEVER delete it, or you open the floodgates for all sorts of problems. ONLY delete references that you are certain belong to your mod.

Already aware of this (and doing this) ^, but thanks for the reminder and thanks for warning/informing other modders :smile:

Edit: Whoops, old post, but what I just wrote above still applies :)
User avatar
Marina Leigh
 
Posts: 3339
Joined: Wed Jun 21, 2006 7:59 pm


Return to V - Skyrim