To BSA or not to BSA

Post » Wed Sep 01, 2010 9:25 pm

The prefix must be either 'MyMod ' (with a space) or 'MyMod-'
So 'MyMod More.bsa', 'MyMod-More.bsa', 'MyMod - More.bsa' will all work.


Gaticus any clear result from your BSA tests so far?

Any easy way to check if my Armamentarium BSA has been loaded properly?

Thanks everyone, I keep learning a lot due your replies:)
User avatar
Mari martnez Martinez
 
Posts: 3500
Joined: Sat Aug 11, 2007 9:39 am

Post » Thu Sep 02, 2010 12:16 am

Gaticus any clear result from your BSA tests so far?

Yes. A BSA cannot override another BSA, as others have said. A BSA can override loose files though.

I have also mostly figured out the BSA load order, I think. It's somewhat complicated. I still need to test more.
Master archives are always loaded first, followed by any other BSA in SArchiveList in the order listed.
Other BSA's are loader in the order of the ESP's timestamp. If one ESP has more than one BSA, those BSA's are loaded in alphabetic order.
The BSA timestamp is used when invalidating loose files.

The interesting bit here is that a BSA can override another BSA only if there is a loose file that invalidates the first BSA. The second BSA can then invalidate the loose file.
Dunno how that could be useful though, really. The only thing I could think of is that BSA-alteration could be used to force precedence, but that presents problems of it's own.
User avatar
Sabrina Schwarz
 
Posts: 3538
Joined: Fri Jul 14, 2006 10:02 am

Post » Wed Sep 01, 2010 10:40 am

So the game is actually paying attention to this?

SMasterMiscArchiveFileName=Oblivion - Misc.bsa
SMasterVoicesArchiveFileName2=Oblivion - Voices2.bsa
SMasterVoicesArchiveFileName1=Oblivion - Voices1.bsa
SMasterSoundsArchiveFileName=Oblivion - Sounds.bsa
SMasterTexturesArchiveFileName1=Oblivion - Textures - Compressed.bsa
SMasterMeshesArchiveFileName=Oblivion - Meshes.bsa


Rather than just this?
SArchiveList=ArchiveInvalidationInvalidated!.bsa, OBMeshes.bsa, OBTextures1.bsa, OBTextures2.bsa, OBTextures3.bsa, Oblivion - Sounds.bsa, Oblivion - Voices.bsa, Oblivion - Misc.bsa, Armamentarium.bsa

User avatar
Ricky Rayner
 
Posts: 3339
Joined: Fri Jul 13, 2007 2:13 am

Post » Thu Sep 02, 2010 1:57 am

How does the BSA invalidate loose files? I assume you checked with loose files that were different than the BSA files but with the same name?

Is it that if an esp uses the BSA files also has a later timestamp than an esp that uses the loose ones?
User avatar
Kevin S
 
Posts: 3457
Joined: Sat Aug 11, 2007 12:50 pm

Post » Wed Sep 01, 2010 11:45 pm

So the game is actually paying attention to this?

Rather than just this?

Yes. This proves it:
SArchiveList=ArchiveInvalidationInvalidated!.bsa, Test-B.bsa, Oblivion - Meshes.bsa, Oblivion - Textures - Compressed.bsa, Oblivion - Sounds.bsa, Oblivion - Voices1.bsa, Oblivion - Voices2.bsa, Oblivion - Misc.bsa

The texture from Test-B.bsa is used. (when there is also a loose file to first invalidate the vanilla texture)
The file creation date of Test-B.bsa was set back to 2005 to ensure that the timestamp wasn't the cause.

Actually, is it paying attention to the MasterArchive settings? I'm not sure, but it does recognize which BSA's are the vanilla ones and loads those first.
Because your BSA's are not named after the MasterArchives, I'm guessing that Oblivion would treat yours without precedence, so my proof would be incorrect for your case.

edit*
I just tested and it doesn't appear that the MasterArchives settings are relevant..
SMasterMiscArchiveFileName=Oblivion - Misc.bsaSMasterVoicesArchiveFileName2=Oblivion - Voices2.bsaSMasterVoicesArchiveFileName1=Oblivion - Voices1.bsaSMasterSoundsArchiveFileName=Oblivion - Sounds.bsaSMasterTexturesArchiveFileName1=Oblivion - Textures - Compressed.bsaSMasterMeshesArchiveFileName=Oblivion - Meshes.bsaSArchiveList=ArchiveInvalidationInvalidated!.bsa, Test-A.bsa, oMeshes.bsa, oTextures - Compressed.bsa, oSounds.bsa, oVoices1.bsa, oVoices2.bsa, oMisc.bsa, Test-B.bsa

I renamed my bsa's, so the MasterArchive settings are referring to invalid files.
Test-A.bsa is being used in this case. If I remove Test-A from the list, then Test-B is used.
So it seems there is some setting within the bsa itself telling oblivion that it's an official archive.
I'll have to rebuild entirely new vanilla BSA's and see if it makes a difference.

How does the BSA invalidate loose files? I assume you checked with loose files that were different than the BSA files but with the same name?

Is it that if an esp uses the BSA files also has a later timestamp than an esp that uses the loose ones?

The precedence is determined by the timestamp of the BSA vs the loose file.
EG. a loose file that replaces some vanilla texture and a BSA with a more recent timestamp that also replaces the vanilla texture.
The loose file invalidates the vanilla texture and the newer BSA overrides the loose file.
If there is no loose file, then the vanilla one would be used because it was never invalidated.

The ESP's timestamp is used to determine the load order of the BSA's themselves, regardless of the BSA's timestamp.
IE, whichever BSA is loaded first will supercede any later loading BSA regardless of the BSA timestamp, unless the first is invalidated by a loose file.
User avatar
Ian White
 
Posts: 3476
Joined: Thu Jul 19, 2007 8:08 pm

Post » Wed Sep 01, 2010 4:03 pm

The ESP's timestamp is used to determine the load order of the BSA's themselves, regardless of the BSA's timestamp.
IE, whichever BSA is loaded first will supercede any later loading BSA regardless of the BSA timestamp, unless the first is invalidated by a loose file.

So what if an esp loads high with an early time stamp but the BSA has a much later time stamp and the files in it compete with a BSA that has an earlier time stamp but the esp associated with it loads later?

Then I've seen some give the advice to make sure and set the timestamp on the BSA via OBMM - I'm not sure if that only effects the vanilla BSA archives or all - if all then that could be problematic - yes or no?

thanks for looking at this.
User avatar
Britta Gronkowski
 
Posts: 3475
Joined: Mon Apr 09, 2007 3:14 pm

Post » Wed Sep 01, 2010 11:10 pm

So what if an esp loads high with an early time stamp but the BSA has a much later time stamp and the files in it compete with a BSA that has an earlier time stamp but the esp associated with it loads later?

Then I've seen some give the advice to make sure and set the timestamp on the BSA via OBMM - I'm not sure if that only effects the vanilla BSA archives or all - if all then that could be problematic - yes or no?

thanks for looking at this.

Then the BSA that is associated with the first loaded ESP is priority. Doesn't matter when the BSA was created, it is loaded first so no other BSA can override the files in it unless the first is invalidated by a newer loose file.
Only then could another BSA override the loose file, if the BSA is newer.

Timestamps do not affect the precedence of the BSA's themselves.
The timestamps do affect invalidation though, when a loose file is present. The advice is correct.
Eg. if you have a modded v1.1 game with loose textures and you upgrade to SI, then the vanilla BSA timestamps will have changed and it will cause the BSA to override the loose files.

Oblivion loads BSA's in this order:
1. MasterArchives
2. Anything else in the SArchiveList, in the order listed
3. BSA's associated with an ESP, in plugin load order. If there are multiple BSA's for one ESP, they are loaded alphabetically.
User avatar
Sudah mati ini Keparat
 
Posts: 3605
Joined: Mon Jul 23, 2007 6:14 pm

Post » Wed Sep 01, 2010 11:44 pm

Seems counter-intuitive that an earlier loading esp'd BSA would override a later loading esp'd BSA. Especially when a later timestamp on loose files wins against the same early loading BSA.

Wonder why that is the case.

[edit] clarified
User avatar
Lindsay Dunn
 
Posts: 3247
Joined: Sun Sep 10, 2006 9:34 am

Post » Wed Sep 01, 2010 9:24 pm

Actually, is it paying attention to the MasterArchive settings? I'm not sure, but it does recognize which BSA's are the vanilla ones and loads those first.


I seem to have answered the question to my satisfaction:
SMasterMiscArchiveFileName=
SMasterVoicesArchiveFileName2=
SMasterVoicesArchiveFileName1=
SMasterSoundsArchiveFileName=
SMasterTexturesArchiveFileName1=
SMasterMeshesArchiveFileName=


The game didn't care one bit about removing those names. Everything loaded according to the SArchiveList setting.
User avatar
mishionary
 
Posts: 3414
Joined: Tue Feb 20, 2007 6:19 am

Post » Wed Sep 01, 2010 11:29 am

'MyMod_someMore.bsa' dosn't work. I just tested.

The prefix must be either 'MyMod ' (with a space) or 'MyMod-'
So 'MyMod More.bsa', 'MyMod-More.bsa', 'MyMod - More.bsa' will all work.

Are you 100% sure? The MMM installation instructions tell people using MMM in FCOM to rename the MMM bsa to
"FCOM_Convergence_Mart's Monster Mod.bsa" to be loaded with the active "FCOM_Convergence.esp", and if this didn't work, it would have been known by now.
User avatar
kristy dunn
 
Posts: 3410
Joined: Thu Mar 01, 2007 2:08 am

Post » Wed Sep 01, 2010 2:18 pm

I've always been led to believe you can't use underscores in BSA names, but they obviously do work or AFK_Weye.bsa would not be loading its assets into the game.
User avatar
Dan Wright
 
Posts: 3308
Joined: Mon Jul 16, 2007 8:40 am

Post » Thu Sep 02, 2010 1:09 am

Ok, that was weird. I just tested again.
No, MyMod_More.bsa will indeed work.

What is weird is that I have Test.esp, Test-A.bsa and Test_B.bsa. To avoid conflict, I renamed Test-A.bsa to Test1-A.bsa thinking that would prevent it from being loaded, but no!
When I tested before, Test1-A was overriding Test_B, so it seemed that Test_B was not loaded but it was just superceded.

It seems that the prefix need only be the name of the ESP. No special rules.
So MyModMore.bsa should work. Or any other name that begins with 'MyMod'. I just tested this too, to be sure and TestA.bsa works.
Maybe in previous versions of Oblivion it is different though.
User avatar
Enny Labinjo
 
Posts: 3480
Joined: Tue Aug 01, 2006 3:04 pm

Post » Wed Sep 01, 2010 2:21 pm

I recall someone else posting about this ... Axil I think and he (if that was who it was) stated that he found it random which got priority between BSA. probably the forum has purged that info.

I'm hoping you nailed down the logic that may lead to better solutions.
User avatar
Nadia Nad
 
Posts: 3391
Joined: Thu Aug 31, 2006 3:17 pm

Post » Wed Sep 01, 2010 10:34 am

Ok, that was weird. I just tested again.
No, MyMod_More.bsa will indeed work.

What is weird is that I have Test.esp, Test-A.bsa and Test_B.bsa. To avoid conflict, I renamed Test-A.bsa to Test1-A.bsa thinking that would prevent it from being loaded, but no!
Test1-A.bsa still starts with the exact name of an active esp, so there's no reason for it to not being loaded. But good, it means that the rule is as general as I thought: As long as the bsa's name starts with the exact, full name of an active bsa, it will be loaded. :)
User avatar
Chad Holloway
 
Posts: 3388
Joined: Wed Nov 21, 2007 5:21 am

Post » Wed Sep 01, 2010 5:25 pm

Gaticus sorry I havent followed this but you might find the following links helpful with research done if you have not seen them before and are furthering the research on the common files in bsa problem.

They are a bit old so anyone new to the subject be careful what you try from these links, some methods are complicated and can be destructive... Redirection as used with Wrye Bash and OBMM is still the best/safest method out of the two methods that work.

Anyway first one you probably know all about the subjects covered here http://devnull.sweetdanger.net/archiveinvalidation.html

Further down on there http://devnull.sweetdanger.net/archiveinvalidation.html#AIExplained

From that point scroll down to Technical Details sub heading, where you will find the conclusions drawn reference common file names in bsa's.

Unfortunately what was linked further on reference all the testing carried out is now lost, but among that lot you may find associations with what you have experienced in testing.

The ArchiveInvalidation problems described here are related to the way Oblivion invalidates entries in its BSA hash tables. In the Oblivion.ini, these are named:

- DirectoryStringTable
- FilenameStringTable

A "hash table" is a very common method of improving performance when you've got a lot of things to keep track of. A directory structure full of files is a perfect candidate for this technology.

Unfortunately, Oblivion's invalidation routines don't seem to be very accurate about how they mark a file as needing to be replaced. It appears there is a "disconnect" in the intersection between these two tables that causes a file invalidated in one path to also get invalidated in all other paths where it exists, resulting in index "collisions" that cause Oblivion to mistakenly think a texture should not be loaded from the BSA.

User avatar
james tait
 
Posts: 3385
Joined: Fri Jun 22, 2007 6:26 pm

Post » Wed Sep 01, 2010 1:33 pm

I recall someone else posting about this ... Axil I think and he (if that was who it was) stated that he found it random which got priority between BSA. probably the forum has purged that info.

I'm hoping you nailed down the logic that may lead to better solutions.

I could understand why it would seem to be random. There are alot of variables and they make it very difficult to do consistent testing.
I have had to redo my tests a dozen times because of little variations that throw a monkey wrench into things leaving all previous tests to question.

If I didn't stumble on the invalidation trick, I might not have sorted it out either. By using invalidation, I was able to create consistency.

Gaticus sorry I havent followed this but you might find the following links helpful with research done if you have not seen them before and are furthering the research on the common files in bsa problem.

Yes, I've read that article and I'm aware of how the invalidation works. Thanks though.
The invalidation problem is caused by hash collisions.

The BSA overlap issue is kind of the reverse problem to be simple.
With invalidation, a loose file causes the hash key for that file in the BSA to be deleted or otherwise ignored.
With BSA overlaps, the hash key for the file in the first BSA prevents the 2nd BSA from loading it's file, because the key already exists.

This is why we can override a BSA with another only if there is a loose file that invalidates the first BSA.
User avatar
loste juliana
 
Posts: 3417
Joined: Sun Mar 18, 2007 7:37 pm

Post » Wed Sep 01, 2010 7:18 pm

The invalidation problem is caused by hash collisions.

The BSA overlap issue is kind of the reverse problem to be simple.
With invalidation, a loose file causes the hash key for that file in the BSA to be deleted or otherwise ignored.
With BSA overlaps, the hash key for the file in the first BSA prevents the 2nd BSA from loading it's file, because the key already exists.

This is why we can override a BSA with another only if there is a loose file that invalidates the first BSA.


Ok, so if you are doing a patch for a mod that has a BSA, you have to release any modified meshes or textures as loose files then, correct?
User avatar
CArlos BArrera
 
Posts: 3470
Joined: Wed Nov 21, 2007 3:26 am

Post » Wed Sep 01, 2010 1:17 pm

Ok, so if you are doing a patch for a mod that has a BSA, you have to release any modified meshes or textures as loose files then, correct?

That seems like the correct conclusion. BSA files should contain unique content only (but can be replaced by loose files in a patch).
User avatar
Anthony Diaz
 
Posts: 3474
Joined: Thu Aug 09, 2007 11:24 pm

Post » Wed Sep 01, 2010 3:51 pm

Theoretically, you might be able to create a BSA patch by having the patched BSA load BEFORE the original BSA.
User avatar
Jamie Moysey
 
Posts: 3452
Joined: Sun May 13, 2007 6:31 am

Post » Wed Sep 01, 2010 10:16 pm

Hello everyone!

I have started comtemplating unpacking all my BSA's to gain greater control over which files are being used in-game. I install everything with BAIN and I'm wondering what the advantages with having files in BSA's really are.

Discussion, input and opinions are welcome.

Hi! :wavey:

BAIN + loose files ftw, absolutely. :) There are no advantages whatsoever to the alternative(s).

Keeps everything simple, clear, and hassle-free. :thumbsup:
User avatar
Queen Bitch
 
Posts: 3312
Joined: Fri Dec 15, 2006 2:43 pm

Post » Wed Sep 01, 2010 7:33 pm

You guys might find this interesting, I just ran into it after unpacking and PyFFI'ing my oblivion - meshes bsa. If the bsa is dated later than loose files the bsa will override them, I'm not sure about the specifics but all of my body mods were overridden by the BSA until I dated it back to 2006.
User avatar
barbara belmonte
 
Posts: 3528
Joined: Fri Apr 06, 2007 6:12 pm

Post » Wed Sep 01, 2010 7:40 pm

You guys might find this interesting, I just ran into it after unpacking and PyFFI'ing my oblivion - meshes bsa. If the bsa is dated later than loose files the bsa will override them, I'm not sure about the specifics but all of my body mods were overridden by the BSA until I dated it back to 2006.


Yup, thats another one of those reasons even us Wrye Bash fanatics keep OBMM around, occasionally it comes in handy with one of its abilities - Reset BSA timestamps.
User avatar
Lizs
 
Posts: 3497
Joined: Mon Jul 17, 2006 11:45 pm

Post » Wed Sep 01, 2010 9:35 am

Small question, is there a limit on how many BSAs can be loaded from a single ESP?

In light of the recent missing meshes and stuff related to BSA loading, I'm thinking of renaming all my BSAs to use DLCShiveringIsles.esp like "DLCShiveringIsles - Armamentarium.bsa" etc to ensure loading.
User avatar
quinnnn
 
Posts: 3503
Joined: Sat Mar 03, 2007 1:11 pm

Post » Wed Sep 01, 2010 5:58 pm

...comes in handy with one of its abilities - Reset BSA timestamps.

I thought I was pretty good with OBMM, but this is one of the features I have been unable to locate so far. I'm using a separate freeware for that, SetFileDate. It was recommended by Psymon and works very well. Need to dig into OBMM once more...
User avatar
Fluffer
 
Posts: 3489
Joined: Thu Jul 05, 2007 6:29 am

Post » Wed Sep 01, 2010 3:07 pm

Utilities -> Archive Invalidation -> Reset BSA Timestamps :)
User avatar
Kayleigh Williams
 
Posts: 3397
Joined: Wed Aug 23, 2006 10:41 am

PreviousNext

Return to IV - Oblivion