Help with a script optimization : IV

» Tue Mar 24, 2015 4:26 am

Hi everyone, I have a little question:

I have a script which is very heavy, at the point it cause a stutter every time it runs.
This script essentially scan the whole cell and compare every "Static" object model path with a set of strings, to return the "type" of cell the player is currently in.
I already set the script to only run on cell change to avoid unnecessary runs, because if the cell is the same, so is the type of cell, obviously.

When the cell change is one of:

interior->interior
interior->exterior
exterior->interior,

the stutter is "hidden" by the loading screen, so no problem.

The problem is with cell changes exterior->exterior, as there's no loading screen and the stutter is noticeable.
The script is this one:

Spoiler

scn DMSGetCellType; Processes cell statics and music type to determine cell typeRef rLocationArray_Var aDataArray_Var aIterTypesArray_Var aIterPathRef rWalkShort iCountShort iMaxShort iMusicTypeShort iCellMusicTypeString_Var sKeyString_Var sTextBegin _Function { rLocation }		Let sText := GetEditorID rLocation	ForEach aIterTypes <- (sv_Split DMSMain.sENUMLOCATIONS, "|~|")[1:-2]	;Check if the cell type is in the location EditorID		If Eval (( sv_Find *aIterTypes, sText ) >= 0 )			SetFunctionValue *aIterTypes			Let iCount := 1			Break		EndIf	Loop	If ( iCount )			;Found the cell type in the EditorID?		sv_Destruct sText		Return	ElseIf Eval !( Player.IsInInterior )			;In exteriors count the trees		Let iCount := GetNumRefs 30 		If Eval ( iCount > 20 )			;With many trees, go on with Forest and stop immediately.			SetFunctionValue "Forest"			Return		EndIf	EndIf		Let aData := ar_Construct "StringMap"	ForEach aIterTypes <- DMSMain.aCellTypesData		Let aData[aIterTypes["Key"]] := 0		;Initialize the keys	Loop	Let rWalk := GetFirstRef 28	While ( rWalk )		Let sText := rWalk.GetModelPath		ForEach aIterTypes <- DMSMain.aCellTypesData	;All cell types			ForEach aIterPath <- (*aIterTypes)[0]		;All substrings for this cell type to search in the model path				Let aData[aIterTypes["Key"]] += (( sv_Find (*aIterPath), sText ) >= 0 )		;<== BOTTLENECK. Count how many models match a cell type			Loop		Loop		Let rWalk := GetNextRef	Loop	Let iCellMusicType := emcGetMusicType 1		;World music	Let sText := "NONE"	ForEach aIterTypes <- aData		Let iCount := (*aIterTypes)		Let sKey := aIterTypes["Key"]		Let iMusicType := DMSMain.aCellTypesData[sKey][1]		Call DMSDebug ("DMS Debug | DMSGetCellType | Type > Count: " + sKey + " > (Music: " + $iMusicType + ") " + $iCount) 		If ( iCount > iMax )			If Eval ( iMusicType < 0 || iMusicType == iCellMusicType )				Let sText := sKey				Let iMax := iCount			EndIf		EndIf	Loop	If Eval ( sText == "Cave" )		If Eval ( ar_HasKey aData "Oblivion" )			If Eval ( aData["Oblivion"] )				SetFunctionValue "Oblivion"				sv_Destruct sText				Return			EndIf		EndIf		If Eval ( ar_HasKey aData "Mine" )			If Eval ( aData["Mine"] )				SetFunctionValue "Mine"				sv_Destruct sText				Return			EndIf		EndIf		SetFunctionValue "Cave"	Else		SetFunctionValue sText	EndIf	sv_Destruct sText	ReturnEnd

As you see, if there's a forest, the script stop immediately, else it will scan the whole cell, to find the cell type, which (example, in an exterior cell) could be a village.
Do you have any idea to make this script lighter, or break it in multiple pieces?
As as I you see, the bottleneck is when it scan all statics model paths (there are MANY models to scan, and must be compared with about about 10-15 possible substrings .

» Tue Mar 24, 2015 12:31 am

Since you only return "Obliviom", "Mine" and "Cave" after the string match, I assume you only want to detect these.

Some general thoughts:

Have you stripped the script ? As it is, you are not setting rLocation and the fist If will not be effective
You could look for "Oblivion" in the wordspace name to avoid the modelpath string match in Oblivion exteriors (or http://cs.elderscrolls.com/index.php?title=IsInOblivion)
Why do you scan exterior cells for "Mine" and "Cave"? Just curious.
You might also, first, count the unique model paths and, then, do the string match, Since the string match is the heavy part, you avoid running it multiple times on the same modelpath

One alternative is to split the load over several frames. What I do when I need this is (1) populate an array in a quest script with data (in you case, the statics you've fonud), (2) have a quest script that anolyses and totalizes a number of those statics per frame and removes them from the array (3) when the array is empty all statics have been anolysed and totalized.

In you case, after the cell change, you run the first part of the script and, if still necessary, populate the quest array with the statics and wait until it is empty. That means the keyword count is finished and you can run the second part of the original script;

» Tue Mar 24, 2015 2:42 am

Ops, there are some bugs, since I just rewrote the script! I fixed them now
- rLocation should be a parameter passed by the calling function.
- There's an "Else, SetFunctionValue sText" at the end! It is intended to return any cell type, not only those 3. That final check ensure "Mine" is returned if "Cave" is the winner but there's a Mine element in the cave (same for Oblivion).

For "IsOblivion", good idea

Essentially, the user define a set of cell types and the relative paths and music type, then the script iterate them all, with no distinction. Mine and Cave are possible type, but the player may define Village as exterior cell type.
Maybe I could make 2 separate arrays, for interior and exterior cells, which should reduce the performance hit a lot.

Do you know some performant way to create an array with unique strings/model paths, and how many times they repeat?
Maybe something like

;For every static xLet Statics[x.GetModelPath] += 1

Could be efficient enough?

And about the multi-frame way: thanks for the tip!

» Tue Mar 24, 2015 7:11 am

Just few optimizations I'm doing:
While in the above script the cell music is checked only after all counts, now I check it before the comparisons, and skip the cell type entirely before wasting CPU. Also, I break the internal cycle after the first match (so performance increase, and the cell type is counted once per static).

While ( rWalk )	Let sText := rWalk.GetModelPath	If Eval ( ar_HasKey DMSMain.aCellStatics sText )		Let DMSMain.aCellStatics[sText] += 1       ;Count the number of statics with the same model and place them in the same key	Else		Let DMSMain.aCellStatics[sText] := 1	EndIf	Let rWalk := GetNextRefLoop...Let aData := ar_Construct "StringMap"ForEach aIterTypes <- DMSMain.aCellTypesData	Let iMusicType := DMSMain.aCellTypesData["Key"][1]	If Eval ( iMusicType >= 0 && iMusicType != iCellMusicType )		Continue	EndIf		Let aData[aIterTypes["Key"]] := 0   ;Initialize counter	ForEach aIterStatics <- DMSMain.aCellStatics		Let sText := aIterStatics["Key"]		ForEach aIterPath <- (*aIterTypes)[0]			If Eval (( sv_Find $(*aIterPath), sText ) >= 0 )		;<== BOTTLENECK				Let aData[aIterTypes["Key"]] += *aIterStatics    ;Sum the number of statics with the same model				Break      ;The cell already match. Stop.			EndIf		Loop	LoopLoop

Also, I would like to truncate the model paths up to the Meshes folder, so the comparisons should be lighter.