[WIP] New merge objects program : III - Morrowind

» Tue Mar 15, 2011 1:13 am

Still appears to be the old version (the EXE has a last modified date of 13th January), doesn't accept "--debug"

Don't understand what's causing that. I've just recompiled and uploaded again. The file's name is changed to esper5.zip just to ensure it's not a problem with the bitbucket website. Please check it says "Beta 4" when you run the program. "--debug" should also be listed in the help when you run esper.exe without any arguments.

» Tue Mar 15, 2011 12:57 am

Yep it's the new one now.

» Tue Mar 15, 2011 12:54 am

Got it now :thumbsup:

Both "Check Resources" and creating a bigmerge.estxt work without errors now.

I think Check Resources isn't scanning BSAs though, here's the reported list of missing resources for Tribunal.esm

I:/Bethesda Softworks/Morrowind/Data Files/Tribunal.esm:    "s\\tx_s_smmn_fabrict.dds"    "m\\tx_soulgem_greater.tga"    "m\\misc_soulgem_greater.nif"    "o\\contain_crate_01.nif"    "f\\furn_crate_open_01.nif"    "d\\in_impsmall_door_01.nif"    "d\\ex_velothi_entrance_01_a.nif"    "a\\tx_chitin_curaiss.tga"    "a\\a_m_chitin_cuirass_gnd.nif"    "a\\tx_chitin_boot.tga"    "a\\a_m_chitin_boot_gnd.nif"    "a\\tx_chitin_greaves.tga"    "a\\a_m_chitin_greaves_gnd.nif"    "a\\tx_chitin_gauntlet.tga"    "a\\a_m_chitin_gauntlet_gnd.nif"    "w\\tx_mace01.tga"    "a\\tx_indoril_helmet.tga"    "a\\a_indoril_m_helmet.nif"    "w\\tx_broadsword_imperial.tga"    "w\\tx_shortsword_imperial.tga"    "d\\ex_nord_door_01.nif"    "d\\ex_imp_loaddoor_02.nif"    "i\\in_v_s_int_corner_02.nif"    "i\\in_moldtrans_00.nif"    "i\\in_moldcave_00.nif"    "i\\in_moldcave_02.nif"    "i\\in_moldcave_03.nif"    "i\\in_moldcave_06.nif"    "i\\in_moldcave_08.nif"    "i\\in_moldcave_09.nif"    "i\\in_moldcave_10.nif"    "i\\in_moldcave_12.nif"    "i\\in_moldcave_13.nif"    "i\\in_moldcave_15.nif"    "i\\in_moldcave_18.nif"    "i\\in_moldcave_21.nif"    "i\\in_moldcave_21_1.nif"    "i\\in_moldcave_25.nif"    "i\\in_moldcave_26.nif"    "i\\in_moldcave_27.nif"    "i\\in_moldcave_doorway00.nif"    "i\\in_moldcave_lroom4_01.nif"    "i\\in_moldcave2_s_00.nif"    "i\\in_moldcave2_s_04.nif"    "i\\in_mudcave_exit00.nif"    "i\\in_mudcave_lroom4_03.nif"    "i\\in_pycave_04.nif"    "i\\in_pycave_lroom4_04.nif"    "d\\door_cavern_doors10.nif"    "r\\minescrib.nif"    "d\\in_r_s_door_01.nif"    "i\\in_redoran_tavern_01.nif"    "d\\ex_emp_tower_01_b.nif"    "d\\hlaalu_loaddoor_ 01.nif"    "d\\hlaalu_loaddoor_ 02.nif"    "r\\slaughterfish.nif"    "d\\in_redoran_hut_door_01.nif"    "d\\ex_t_door_01.nif"    "m\\tx_book_open_03.tga"    "m\\tx_octavo_03.tga"    "m\\misc_com_bottle_02.tga"    "m\\misc_com_bottle_09.tga"    "m\\misc_com_bottle_10.tga"    "m\\misc_com_bottle_11.tga"    "m\\misc_com_bottle_13.tga"    "m\\misc_com_bottle_14.tga"    "m\\misc_com_bottle_15.tga"    "d\\ex_t_door_stone_large.nif"    "w\\tx_steel_battleaxe.tga"    "w\\tx_warhammer.tga"    "w\\tx_broadsword_iron.tga"    "w\\tx_mace_iron.tga"    "w\\tx_silver_staff.tga"    "w\\tx_silver_waraxe.tga"    "o\\contain_chest_small_02.nif"    "o\\contain_chest_small_02.nif"    "o\\contain_chest_small_02.nif"    "o\\contain_chest_small_01.nif"    "o\\contain_chest_small_01.nif"    "o\\contain_chest_small_01.nif"    "a\\tx_netch_pauldron.tga"    "a\\a_netch_m_pauldron_gnd.nif"    "a\\tx_netch_cuirass2.tga"    "a\\a_netch_m_currais2_gnd.nif"    "a\\tx_netch_gauntlet.tga"    "a\\a_netch_m_gauntlet_gnd.nif"    "a\\tx_netch_gauntlet.tga"    "a\\a_netch_m_gauntlet_gnd.nif"    "a\\tx_netch_boots.tga"    "a\\a_netch_m_boot_gnd.nif"    "a\\tx_nordicfur_helmet.tga"    "a\\a_nordicfur_helmet.nif"    "a\\tx_nordicfur_pauldron.tga"    "a\\a_nordicfur_pauldron_gnd.nif"    "a\\tx_nordicfur_boots.tga"    "a\\a_nordicfur_boot_gnd.nif"    "a\\tx_studdedleather_cuirass.tga"    "a\\a_studdedleather_c_gnd.nif"    "a\\tx_imperialchain_cuirass.tga"    "a\\a_m_imperialchain_c_gnd.nif"    "a\\tx_steel_helmet.tga"    "a\\a_steel_helmet.nif"    "a\\tx_steel_curaiss.tga"    "a\\a_steel_cuirass_gnd.nif"    "a\\tx_steel_gauntlet.tga"    "a\\a_steel_gauntlet_gnd.nif"    "a\\tx_steel_gauntlet.tga"    "a\\a_steel_gauntlet_gnd.nif"    "a\\tx_iron_cuirass.tga"    "a\\a_iron_cuirass_gnd.nif"    "a\\tx_bonemold_boots.tga"    "a\\a_bonemold_boot_gnd.nif"    "a\\tx_bonemold_shield.tga"    "a\\shield_bonemold.nif"    "a\\tx_trollbone_helm.tga"    "a\\a_trollbone_helm.nif"    "a\\tx_shield_dreugh.tga"    "a\\shield_dreugh.nif"    "a\\tx_netch_pauldron.tga"    "a\\a_netch_m_pauldron_gnd.nif"    "a\\tx_nordicfur_pauldron.tga"    "a\\a_nordicfur_pauldron_gnd.nif"    "c\\tx_shoes_expensive_1.tga"    "c\\c_m_shoe_expens_1_gnd.nif"    "n\\tx_local_brew_01.tga"    "n\\potion_local_brew_01.nif"    "o\\contain_com_sack_02.nif"    "d\\ex_t_door_sphere_01.nif"    "f\\active_de_bed_01.nif"    "f\\active_de_bed_06.nif"    "f\\active_de_bed_18.nif"    "f\\active_de_bed_23.nif"    "o\\contain_de_closet_01.nif"    "m\\misc_de_bowl_01.tga"    "m\\misc_de_bowl_01.nif"    "m\\misc_de_goblet_02.tga"    "m\\misc_de_goblet_04.tga"    "m\\misc_de_pitcher_01.tga"    "f\\furn_com_torch_ring_01.nif"    "o\\contain_crate_02.nif"    "m\\misc_com_pillow_01.tga"    "m\\misc_com_pillow_01.nif"    "f\\furn_rug_big_06.nif"    "m\\tx_pot_glass_peach_02.tga"    "m\\misc_pot_glass_peach_02.nif"    "m\\tx_bowl_bugdesign_01.tga"    "m\\misc_bowl_bugdesign_01.nif"    "m\\tx_bowl_glass_peach_01.tga"    "m\\misc_bowl_glass_peach_01.nif"    "m\\tx_pot_blue_01.tga"    "m\\tx_pot_blue_02.tga"    "m\\misc_cloth00.tga"    "m\\misc_foldedcloth00.nif"    "m\\misc_de_bowl_white_01.tga"    "f\\furn_com_bookshelf_01.nif"    "m\\misc_redware_cup.tga"    "m\\misc_redware_cup.nif"    "m\\misc_redware_vase.tga"    "m\\misc_redware_vase.nif"    "a\\tx_imperialchain_greaves.tga"    "a\\a_m_imperialchain_gr_gnd.nif"    "a\\tx_imperial_curaiss.tga"    "a\\a_imperial_m_cuirass_gnd.nif"    "o\\contain_com_sack_01.nif"    "o\\contain_de_chest_01.nif"    "i\\in_vs_pitfloor_01.nif"    "o\\contain_chest_small_02.nif"    "i\\in_dwrv_corr1_00.nif"    "i\\in_hlaalu_room_entry.nif"    "i\\in_hlaalu_room_side.nif"    "d\\in_hlaalu_door.nif"    "r\\sphere_centurions.nif"    "r\\g_centurionspider.nif"    "r\\steam_centurions.nif"    "r\\rust rat.nif"    "o\\contain_com_basket_01.nif"    "o\\contain_com_sack_02.nif"    "f\\furn_de_bar_01.nif"    "f\\furn_de_bar_03.nif"    "f\\furn_de_bar_04.nif"    "o\\contain_de_desk_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_closet_01.nif"    "x\\ex_hlaalu_fb_01.nif"    "o\\contain_crate_01.nif"    "i\\in_hlaalu_ashpit_01.nif"    "o\\contain_urn_02.nif"    "o\\contain_urn_04.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "d\\in_t_s_plain_door.nif"    "a\\tx_imperialchain_pauldron.tga"    "a\\a_m_imperialchain_pa_gnd.nif"    "a\\tx_imperialchain_pauldron.tga"    "a\\a_m_imperialchain_pa_ua.nif"    "a\\tx_iron_helm_01.tga"    "a\\a_iron_helm_01.nif"    "o\\contain_crate_01.nif"    "x\\ex_de_shack_door.nif"    "i\\in_de_shack_door.nif"    "f\\furn_dwrv_steam_00.nif"    "f\\furn_dwrv_tranny00.nif"    "f\\furn_dwrv_beltdrive00.nif"    "f\\furn_dwrv_dynamo00.nif"    "f\\furn_dwrv_fitting40.nif"    "o\\contain_dwrv_closet00.nif"    "f\\furn_dwrv_bookshelf00.nif"    "m\\misc_skull00.tga"    "m\\misc_skull00.nif"    "x\\ex_de_shack_steps.nif"    "f\\furn_de_ex_bench_01.nif"    "f\\furn_shell20.nif"    "d\\ex_cave_door_01.nif"    "o\\contain_chest_small_02.nif"    "x\\ex_dae_ruin_stair01_short.nif"    "i\\in_dae_room_l_side_01.nif"    "r\\ashghoul.nif"    "r\\ancestorghost.nif"    "f\\furn_6th_bell1.nif"    "f\\furn_6th_bell2.nif"    "f\\furn_6th_bell3.nif"    "f\\furn_6th_bell4.nif"    "l\\light_com_chandelier_05.nif"    "l\\light_com_chandelier_05.nif"    "l\\light_com_chandelier_05.nif"    "l\\light_com_chandelier_05.nif"    "f\\furn_chandelier_burnedout.nif"    "f\\furn_chandelier_burnedout.nif"    "f\\furn_chandelier_burnedout.nif"    "f\\furn_chandelier_burnedout.nif"    "w\\w_nerevar_blade_fire.dds"    "w\\w_nerevarblade_flame.nif"    "w\\tx_w_nerevarbladeleft.dds"    "w\\w_nerevarbladeleft.nif"    "w\\tx_w_nerevarbladeright.dds"    "w\\w_nerevarbladeright.nif"    "l\\light_com_lamp_01.nif"    "l\\light_com_lamp_01.nif"    "l\\light_com_lamp_01.nif"    "l\\light_com_lamp_01.nif"    "l\\light_lamp_burnedout.nif"    "l\\light_lamp_burnedout.nif"    "l\\light_lamp_burnedout.nif"    "l\\light_lamp_burnedout.nif"    "d\\in_mh_door_02.nif"    "w\\tx_dagger_daedric.tga"    "editormarker.nif"    "i\\in_lava_blacksquare.nif"    "x\\ex_mh_pavillion_gate.nif"    "x\\ex_mh_pavillion_gate_02.nif"    "x\\ex_mh_pavillion_wall_e.nif"    "x\\ex_mh_pavillion_wall_w.nif"    "x\\ex_mh_pav_fountain.nif"    "x\\ex_mh_pav_walk_n.nif"    "x\\ex_mh_pav_walk_s.nif"    "x\\ex_mh_pav_land_e.nif"    "x\\ex_mh_pav_land_w.nif"    "r\\fabricant_hulking.nif"    "r\\fabricant_hulking.nif"    "r\\fabricant.nif"    "x\\ex_mh_pav_gate_door.nif"    "x\\ex_mh_pav_tower.nif"    "x\\ex_mh_almalexia_whole.nif"    "x\\ex_mh_pav_crack.nif"    "x\\ex_mh_pav_rubble_01.nif"    "x\\ex_mh_pav_bstatue_01.nif"    "x\\ex_mh_pav_bstatue_02.nif"    "x\\ex_mh_pav_bstatue_03.nif"    "x\\ex_mh_pav_bstatue_04.nif"    "x\\ex_mh_pav_bstatue_05.nif"    "x\\ex_mh_pav_bstatue_06.nif"    "x\\ex_mh_pav_bstatue_07.nif"    "x\\ex_mh_pav_bstatue_08.nif"    "x\\ex_mh_pav_bstatue_09.nif"    "x\\ex_mh_pav_bstatue_10.nif"    "x\\ex_mh_pav_bstatue_11.nif"    "x\\ex_mh_pav_bstatue_12.nif"    "x\\ex_mh_pav_bstatue_13.nif"    "x\\ex_mh_pav_bstatue_14.nif"    "x\\ex_mh_pav_bstatue_15.nif"    "x\\ex_mh_pav_top.nif"    "l\\ex_mh_streetlamp_01.nif"    "a\\a_nerevar_b_shield.dds"    "a\\a_dwemer_shield_battle.nif"    "r\\dremora.nif"    "c\\tx_robe_white.dds"    "c\\c_robe_white_gnd.nif"    "c\\c_robe_white.nif"    "x\\ex_mh_pav_ladder.nif"    "w\\tx_w_adtium_shortsword.dds"    "w\\w_adamantium_shortsword.nif"    "a\\tx_almindoril_boots.tga"    "a\\a_almindoril_boot_gnd.nif"    "a\\tx_almindoril_cuirass.tga"    "a\\a_almindoril_cuirass_gnd.nif"    "a\\tx_almindoril_gauntlet.tga"    "a\\a_almindoril_gauntlet_gnd.nif"    "a\\tx_almindoril_gauntlet.tga"    "a\\a_almindoril_gauntlet_gnd.nif"    "a\\tx_almindoril_greaves.tga"    "a\\a_almindoril_greaves_gnd.nif"    "a\\tx_almindoril_helmet.tga"    "a\\a_almindoril_helmet.nif"    "a\\tx_almindoril_pauldron.tga"    "a\\a_almindoril_pauldron_gnd.nif"    "a\\tx_almindoril_pauldron.tga"    "a\\a_almindoril_pauldron_gnd.nif"    "a\\tx_shield_almindoril.dds"    "a\\shield_amindoril.nif"    "editormarker.nif"    "c\\c_m_robe_liche.nif"    "c\\c_m_helseth_robe.nif"    "a\\a_dwemer_shield_battle.nif"    "x\\ex_mh_palace_gate.nif"    "i\\in_mh_pav_lava_cave.nif"    "i\\in_mh_pav_ladder.nif"    "x\\ex_mh_pav_water_spray.nif"    "x\\ex_mh_palace_gate.nif"    "x\\ex_mh_pav_spout.nif"    "x\\ex_mh_pav_steps.nif"    "i\\in_m_sewer_drain_01.nif"    "m\\tx_paper_roll_01.dds"    "m\\text_paper_roll_01.nif"    "i\\act_dwe_powerbox00.nif"    "m\\misc_dwrv_artifact50.tga"    "editormarker.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "o\\contain_de_chest_01.nif"    "r\\goblin02.nif"    "r\\durzog_collar.nif"    "f\\furn_ashl_chime_01.nif"    "editormarker.nif"    "editormarker.nif"    "editormarker.nif"    "editormarker.nif"    "in_om_shrinefx.nif"    "i\\in_mh_temple_02.nif"    "i\\in_mh_temple_hall_01.nif"    "f\\flora_treedead_mh_01.nif"    "f\\flora_treedead_mh_02.nif"    "c\\tx_mournhold_skirt.dds"    "c\\c_skirt_mournhold_gnd.nif"    "c\\c_f_shirt_common2_skin.nif"    "c\\c_f_shirt_common1_skin.nif"    "m\\tx_paper_roll_01.tga"    "m\\text_paper_roll_01.nif"    "f\\flora_tree_mh_01.nif"    "f\\flora_tree_mh_02.nif"    "f\\flora_treedead_mh_01.nif"    "f\\flora_treedead_mh_02.nif"    "o\\contain_com_chest_02.nif"    "m\\misc_dwrv_artifact50.tga"    "d\\in_mh_door_02.nif"    "d\\ex_mh_temple_door_01.nif"    "r\\rust rat.nif"    "r\\rust rat.nif"    "r\\skeleton.nif"    "m\\tx_scroll_open_01.tga"    "m\\text_scroll_01.nif"    "o\\contain_crate_01.nif"    "m\\tx_note_01.tga"    "m\\text_note_01.nif"    "o\\contain_crate_01.nif"    "m\\tx_note_01.tga"    "m\\text_note_01.nif"    "a\\a_adamantium_helm.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "i\\act_sotha_red.nif"    "i\\act_sotha_red.nif"    "i\\act_sotha_green.nif"    "i\\act_sotha_green.nif"    "i\\act_sotha_steam00.nif"    "i\\in_om_waterfall_small.nif"    "o\\contain_com_chest_02.nif"    "w\\tx_silver_dagger.tga"    "w\\tx_silver_dagger.tga"    "w\\tx_silver_dagger.tga"    "m\\tx_octavo_02.tga"    "d\\in_mh_door_02.nif"    "w\\tx_longspear_ebony.tga"    "r\\azura.nif"    "c\\tx_amulet_extrav2.tga"    "c\\amulet_extravagant_2.nif"    "o\\contain_com_chest_02.nif"    "m\\tx_key_standard_01.tga"    "m\\key_standard_01.nif"    "w\\tx_w_goblin_shield.dds"    "w\\w_goblin_shield.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "editormarker_box_01.nif"    "a\\tx_adamantium_helm.dds"    "a\\a_adamantium_helm.nif"    "r\\durzog.nif"    "editormarker.nif"    "o\\contain_dwrv_chest10.nif"    "editormarker.nif"    "i\\act_sotha_fabsteam.nif"    "d\\ex_mh_door_02.nif"    "d\\in_mh_door_02.nif"    "i\\act_sotha_steam00.nif"    "w\\tx_dagger_glass.tga"    "w\\tx_dagger_glass.tga"    "i\\act_sotha_collide00.nif"    "r\\rust rat.nif"    "l\\light_mh_rope_lantern.nif"    "x\\ex_mh_lantern_post.nif"    "box.nif"    "d\\ex_mh_door_02.nif"    "m\\tx_key_standard_01.tga"    "m\\key_standard_01.nif"    "f\\active_mh_forcefield.nif"    "f\\active_mh_forcefield.nif"    "f\\active_mh_forcefield.nif"    "f\\active_mh_forcefield.nif"    "f\\active_mh_forcefield.nif"    "f\\active_mh_forcefield.nif"    "w\\tx_silver_dagger.tga"    "r\\fabricant_hulking.nif"    "i\\broken_imperfect.nif"    "c\\c_m_robe_liche.1st.nif"    "c\\c_m_helseth_robe.1st.nif"    "c\\c_robe_white.1st.nif"    "l\\light_de_lantern_03.nif"    "d\\in_mh_door_02.nif"    "d\\in_mh_door_02.nif"    "m\\tx_fabricant_elixir.dds"    "m\\misc_fabricant_elixir.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"    "o\\contain_de_chest_01.nif"

I don't think that's right.

Looking through what happens with merging objects: I think I need to create some kind of test plugins.

Have a look at this:

Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #>

"Original:" is Millinghordesman's Clear Glass Equipment, "New:" is Alaisaigae's Left Gloves. "Merged:" just seems to be taking all the info from Left Gloves as it loads later. I'd have expected Clear Glass Equipment's Model info to be used and the Icon from Left Gloves.

» Mon Mar 14, 2011 8:23 pm

eeeickythump - glad to see someone tackling this! :goodjob:

I regret I do not have any useful coding comments to offer but I did want to give this project a boost. I have found TESTool easier to use than some but it has its challenges and a replacement that worked reliably and error free would be most welcome.
In that regard, I would echo the comments made earlier by DarkDiva; it needs to be easy to use. 'Just Fix It' may be not the right thing for which to ask but a GUI or other very user friendly interface would help a great deal, even if someone else had to put that part together.

Good luck with this. It is most welcome indeed.

» Mon Mar 14, 2011 3:04 pm

Got it now :thumbsup:

Thank god for that. I think the lisp compiler was saving the executable in different directory from where I thought -- hence I was uploading an old exe thinking it was freshly compiled.

Both "Check Resources" and creating a bigmerge.estxt work without errors now.

I think Check Resources isn't scanning BSAs though, here's the reported list of missing resources for Tribunal.esm

I think you're right. It does scan BSAs, but something in the "filename matching" logic isn't working.

Have a look at this:

Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #>

"Original:" is Millinghordesman's Clear Glass Equipment, "New:" is Alaisaigae's Left Gloves. "Merged:" just seems to be taking all the info from Left Gloves as it loads later. I'd have expected Clear Glass Equipment's Model info to be used and the Icon from Left Gloves.

(I see it's not updating the length of merged string fields correctly - the latter record's ITEX is truncated)

Re the above, I'm not sure if what you're expecting is quite right.

If you are merging only 2 mods, then I would expect the records in the "newer" mod to essentially overwrite the older. Icon ("ITEX") from "Left Gloves" is used because it's newer. Similarly, model ("MODL") from "Left Gloves" is also used -- again because "Left Gloves" value is newer and so overrides "Clear Glass".

The logic is that whenever a unique record (eg "glass_pauldron_right") is encountered for the first time in a list of ESPs/ESMs, a copy of that record is remembered as "original". In addition, a working copy of the record called "merged" is kept, which starts out the same as "original".

Then, each time a different version of the record is encountered in subsequent plugins, each value in the "new" record is compared to the value in the "original" record. Whenever the values differ, the "new" value is written into "merged", overwriting any previously stored value.

After all plugins have been scanned, all the "merged" records that have been altered, are written to the output file.

(There are a few record types which are a bit more complex, like containers, beause they need to aggregate subrecords rather than overwriting.)

So, the best way to test "merge-records" is to merge 3 files. The first contains an "original" of the record (in practice this usually comes from Morrowind/Tribunal/Bloodmoon.esm). The other two each modify that same record in *different* ways.
For example, suppose:

the base stats of "glass_pauldron_right" in Morrowind.esm were health 500, value 9600, and
"Clear Glass" sets armour value to 15000, and
"Left Gloves" sets armour health to 200
You run: esper -r Morrowind.esm "Clear Glass.esp" "Left Gloves.esp"

Then the resulting merged plugin should have armour with value 15000 (from clear glass) and health 200 (from left gloves), rather than both health and value set to "Left Gloves" values, which is what you effectively get if you just load your plugins in the above order without merging objects.

» Mon Mar 14, 2011 8:54 pm

eeeickythump - glad to see someone tackling this! :goodjob:

I regret I do not have any useful coding comments to offer but I did want to give this project a boost. I have found TESTool easier to use than some but it has its challenges and a replacement that worked reliably and error free would be most welcome.
In that regard, I would echo the comments made earlier by DarkDiva; it needs to be easy to use. 'Just Fix It' may be not the right thing for which to ask but a GUI or other very user friendly interface would help a great deal, even if someone else had to put that part together.

Good luck with this. It is most welcome indeed.

Thanks for the encouragement!

For the time being I won't be writing a GUI, as I would have to teach myself GUI programming to do so. (I have written a windowing GUI for console-based programs, intended for use with roguelike games -- I could use that I suppose.)

In the interim it could be packaged with 2 batch files. One writes names of all active plugins to a file. User edits file in text editor to remove plugins they don't want to merge. Then the second batch file runs esper using the text file to get names of plugins to merge. Not a GUI but not too difficult, just click on the batch files and edit a text file.

The other idea would be if, once esper is more tested and mature, the Wrye Mash maintainers wanted to add a dialog box or a little button to the bottom of their program to launch it, like they have with MWSE, BOSS, etc.

That just gave an idea -- users could use Wrye Mash to put a tag in the description field {{NoMerge}} or {{NoEsper}}. Esper could then automatically skip these plugins when merging.

» Tue Mar 15, 2011 12:53 am

Doing that could finally bring Wrye Mash somewhat closer to Wrye Bash in functionality with the Bashed Patch features. Wrye Mash could output the esp list to Esper and Esper would merge the objects and then it would show up in Wrye Mash after it's done.

PS. If you want to go further you could implement the ability to choose what types of records from each mod gets included in the Merged Objects.esp. If it gets included in Wrye Mash then they could use something similar to Bashed Tags to determine what objects get merged. If it doesn't you could have parameters in the esp list that a person could set to select the types of records to merge.

PPS. The above would solve the problem of some mods having parts that don't work well with being merged and you would still be able to merge the rest of the objects in the mod.

» Mon Mar 14, 2011 10:20 pm

The other idea would be if, once esper is more tested and mature, the Wrye Mash maintainers wanted to add a dialog box or a little button to the bottom of their program to launch it, like they have with MWSE, BOSS, etc.

I am in no way a maintainer, but tempted to have a look at doing it. I just have to see if I can get Mash Working on Linux.

That just gave an idea -- users could use Wrye Mash to put a tag in the description field {{NoMerge}} or {{NoEsper}}. Esper could then automatically skip these plugins when merging.

Good idea. However, given the limit on the size of the description (I don't think it is variable length?), maybe just have it {{NM}} or {{NE}} or {{!Esper}} or {{!Merge}}. Short as possible IMHO.

» Mon Mar 14, 2011 4:52 pm

Now I'm finally able too use Better Heads, Better clothes and PTE without any NPC conflict and without modifying either of these mods. I'll test this tool later and write back if I have further question or issues. A GUI like in the old TESTool with options on what types of records from each mod too parses would be very handy, but if this little program are able too merge the clothes object from Better clothes with PTE it's all nice for the time being. Good luck with your further development of this awesome successor too TESTool.

» Mon Mar 14, 2011 8:30 pm

Thank god for that. I think the lisp compiler was saving the executable in different directory from where I thought -- hence I was uploading an old exe thinking it was freshly compiled.

Glad it's sorted

The logic is that whenever a unique record (eg "glass_pauldron_right") is encountered for the first time in a list of ESPs/ESMs, a copy of that record is remembered as "original". In addition, a working copy of the record called "merged" is kept, which starts out the same as "original".

Then, each time a different version of the record is encountered in subsequent plugins, each value in the "new" record is compared to the value in the "original" record. Whenever the values differ, the "new" value is written into "merged", overwriting any previously stored value.

OK, I think I understand how that goes. When generating that information I didn't specify filenames, I was relying on morrowind.ini. To show what I was expecting to happen here's the equivalent part of my TESTool.log:

ARMO glass_pauldron_right    > Morrowind.esm        MODL      a\A_Glass_Pauldron_GND.nif        FNAM      Glass Right Pauldron        AODT      Type=3, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      a\TX_Glass_pauldron.tga        INDX/BNAM 23 / a_glass_cl_pauldron        INDX/BNAM 13 / a_glass_pauldron_ua        INDX/BNAM 11 / a_glass_pauldron_fa> Clear Glass.esp        MODL      MH\a\clear_glass_pldrn_GND.NIF        FNAM      Glass Right Pauldron        AODT      Type=3, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      a\TX_Glass_pauldron.tga        INDX/BNAM 23 / a_glass_cl_pauldron        INDX/BNAM 13 / a_glass_pauldron_ua        INDX/BNAM 11 / a_glass_pauldron_fa> LeftGloves_Addon_v2.esp        MODL      a\A_Glass_Pauldron_GND.nif        FNAM      Glass Right Pauldron        AODT      Type=3, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      Lefty\TX_Glass_pauldron_R.tga        INDX/BNAM 23 / a_glass_cl_pauldron        INDX/BNAM 13 / a_glass_pauldron_ua        INDX/BNAM 11 / a_glass_pauldron_fa----------------------------        MODL      MH\a\clear_glass_pldrn_GND.NIF        FNAM      Glass Right Pauldron        AODT      Type=3, Weight=3.00, Value=9600, HP=500, Ench=15, AC=50        ITEX      Lefty\TX_Glass_pauldron_R.tga        INDX/BNAM 23 / a_glass_cl_pauldron        INDX/BNAM 13 / a_glass_pauldron_ua        INDX/BNAM 11 / a_glass_pauldron_fa

It looks like TESTool is doing a similar thing but the "original" record (i.e. the one encountered for the first time) is the record in Morrowind.esm (in this instance but it could be any of the Bethesda ESMs depending on the object) and the changes made by subsequent plugins (in this case "Clear Glass.esp" and "LeftGloves_Addon_v2.esp") are compared against that original record (i.e. the Morrowind.esm one) and written into "merged".

There are cases when TESTool can't merge an object, like this:

ARMO glass_pauldron_left    > Morrowind.esm        MODL      a\A_Glass_Pauldron_GND.nif        FNAM      Glass Left Pauldron        AODT      Type=2, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      a\TX_Glass_pauldron.tga        INDX/BNAM 24 / a_glass_cl_pauldron        INDX/BNAM 14 / a_glass_pauldron_ua        INDX/BNAM 12 / a_glass_pauldron_fa> Clear Glass.esp        MODL      MH\a\clear_glass_pldrn_GND.NIF        FNAM      Glass Left Pauldron        AODT      Type=2, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      a\TX_Glass_pauldron.tga        INDX/BNAM 24 / a_glass_cl_pauldron        INDX/BNAM 14 / a_glass_pauldron_ua        INDX/BNAM 12 / a_glass_pauldron_fa> LeftGloves_Addon_v2.esp        MODL      Lefty\A_Glass_Pauldron_L.nif        FNAM      Glass Left Pauldron        AODT      Type=2, Weight=3.00, Value=http://forums.bethsoft.com/index.php?/topic/1067443-wip-new-merge-objects-program/9600, HP=500, Ench=15, AC=50        ITEX      Lefty\TX_Glass_pauldron_L.tga        INDX/BNAM 24 / a_glass_cl_pauldron        INDX/BNAM 14 / a_glass_pauldron_ua        INDX/BNAM 12 / a_glass_pauldron_fa----------------------------Some information has been lost during merging due to a conflictResult discarded due to being identical to the last loaded object

In this instance the changes made by Clear Glass (just to MODL) are discarded as the later-loading Left Gloves also changes MODL. There's only so much that can be done and the basic load order principle of most important plugins load later is followed by TESTool.

So, the best way to test "merge-records" is to merge 3 files. The first contains an "original" of the record (in practice this usually comes from Morrowind/Tribunal/Bloodmoon.esm). The other two each modify that same record in *different* ways.
For example, suppose:

* the base stats of "glass_pauldron_right" in Morrowind.esm were health 500, value 9600, and
* "Clear Glass" sets armour value to 15000, and
* "Left Gloves" sets armour health to 200
* You run: esper -r Morrowind.esm "Clear Glass.esp" "Left Gloves.esp"

Then the resulting merged plugin should have armour with value 15000 (from clear glass) and health 200 (from left gloves), rather than both health and value set to "Left Gloves" values, which is what you effectively get if you just load your plugins in the above order without merging objects.

Thought I'd try this, ran the command and piped it to a text file:

Esper, version Beta 4Released under GPL; see the file LICENSE for details.Command line arguments received:    ("-r" "Morrowind.esm" "Clear Glass.esp" "LeftGloves_Addon_v2.esp")NIL Clear Glass.esp .LeftGloves_Addon_v2.esp Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #>Merged records:Original:#   #   #   #   #   #   #   #   #   #   #>New:#   #   #   #   #   #   #   #   #   #   #>Merged:#   #   #   #   #   #   #   #   #   #>..Merged records:"ARMO": "glass_pauldron_left""ARMO": "glass_pauldron_right"2

So, despite specifying Morrowind.esm its contents aren't taken as the "original" during merging.

Finally, I think some tweaking to the file header of "output.esp" would be in order. You could have Author as "Esper generated file. Esper created by eeeickythump", just to stoke your ego

Perhaps "description" could be driven by whether -r or -p is used (maybe "merged objects" for the former and "merged plugins" for the latter).

The Esper-generated plugin's file masters information needs to be correctly set. Currently the only dependency set is against Morrowind.esm; it might be worthwhile to set Bloodmoon.esm and Tribunal.esm too if present. However I'm in two minds about that as there's no way to work out whether the expansions are there if someone is just merging a couple of plugins and not including the Bethesda ESMs. Not too big a deal.

More seriously, however, is that when recording the Morrowind.esm dependency Esper records the filesize of Morrowind.esm as 1234 bytes. This'll lead to the "One or more plugins could not find the correct versions of the master files they depend on. Errors may occur during load or game play" error message. Easy enough to fix in Wrye Mash but it could cause needless concern for people using Esper. Not sure how easy it is to get the Morrowind.esm filesize and use that?

» Mon Mar 14, 2011 8:11 pm

It seems that esper has a lot of problems with certain scripts and enchantments:

One of the files that "merged-objects.esp" is dependent on has changed since the last save.This may result in errors. Saving again will clear this messagebut not necessarily fix any errors.One or more plugins could not find the correct versions of the master files they depend on. Errors may occur during load or game play. Check the "Warnings.txt" file for more information.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_01'.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_02'.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_03'.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_04'.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_05'.Unable to find script 'MU_GH_Sounds_Muckspun' on object 'flora_muckspunge_06'.Unable to find script 'MU_GH_Sounds_Heat' on object 'flora_heather_01'.Unable to find script 'MU_GH_Sounds_Gold' on object 'flora_gold_kanet_01'.Unable to find script 'MU_GH_Sounds_Ston' on object 'flora_stoneflower_01'.Unable to find script 'MU_GH_Sounds_Will' on object 'flora_willow_flower_01'.Unable to find script 'MU_GH_Sounds_Tramaro' on object 'tramaroot_01'.Unable to find script 'MU_GH_Sounds_Tramaro' on object 'tramaroot_06'.Unable to find script 'MU_GH_Sounds_Tramaro' on object 'tramaroot_02'.Unable to find script 'MU_GH_Sounds_Tramaro' on object 'tramaroot_03'.Unable to find script 'MU_GH_Sounds_Tramaro' on object 'tramaroot_04'.Unable to find script 'MU_GH_Sounds_Tram' on object 'tramaroot_05'.Unable to find script 'MU_GH_Sounds_RM_Scat' on object 'flora_rm_scathecraw_01'.Unable to find script 'MU_GH_Sounds_RM_Scat' on object 'flora_rm_scathecraw_02'.Unable to find script 'MU_GH_Sounds_Wick' on object 'flora_wickwheat_01'.Unable to find script 'MU_GH_Sounds_Wick' on object 'flora_wickwheat_02'.Unable to find script 'MU_GH_Sounds_Wick' on object 'flora_wickwheat_03'.Unable to find script 'MU_GH_Sounds_Wick' on object 'flora_wickwheat_04'.Unable to locate Enchanting 'viper_enuss_EN'.Unable to locate Enchanting 'dynamic horny fist_ects.esp)'.Unable to locate Enchanting 'dynamic horny fist_N'.Unable to find script 'MU_GH_Sounds_Kres' on object 'flora_kreshweed_01'.Unable to find script 'MU_GH_Sounds_Kres' on object 'flora_kreshweed_02'.Unable to locate Enchanting 'dynamic horny fist_eetals'.Unable to locate Enchanting 'dynamic horny fist_ects.esp)'.Unable to locate Enchanting 'dynamic horny fist_'.Unable to locate Enchanting 'dynamic horny fist__01'.Unable to find script 'MU_GH_Sounds_Ston' on object 'flora_stoneflower_02'.Unable to find script 'MU_GH_Sounds_Gold' on object 'flora_gold_kanet_02'.Unable to find script 'MU_GH_Sounds_Will' on object 'flora_willow_flower_02'.Unable to find script 'MU_GH_Sounds_Blac' on object 'flora_black_anther_01'.Unable to find script 'MU_GH_Sounds_Blac' on object 'flora_black_anther_02'.Unable to find script 'MU_GH_Sounds_Fire_Fe' on object 'flora_fire_fern_01'.Unable to find script 'MU_GH_Sounds_Fire_Fe' on object 'flora_fire_fern_02'.Unable to find script 'MU_GH_Sounds_Fire_Fe' on object 'flora_fire_fern_03'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_01'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_02'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_03'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_04'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_05'.Unable to find script 'MU_GH_Sounds_Trama_S' on object 'contain_trama_shrub_06'.Unable to find script 'MU_GH_Sounds_BC_F' on object 'flora_bc_fern_01'.Unable to locate Enchanting 'AIM_Auriels_shieen'.Unable to locate Enchanting 'masterful horny fistund'.Unable to locate Enchanting 'shard_enuster_01'.Texture "Textures\_land_default.tga" count 3.Texture "Textures\menu_thick_border_bottom_right_corner.dds" count 2.

Game files:

[Game Files]GameFile0=Morrowind.esmGameFile1=Tribunal.esmGameFile2=Bloodmoon.esmGameFile3=Morrowind Patch v1.6.5-BETA.esmGameFile4=Bloodmoon Landscape Overhaul 1.0.esmGameFile5=Texture Fix 1.8.esmGameFile6=TR_Data.esmGameFile7=TR_Map1.esmGameFile8=TR_Map2.esmGameFile9=VGreetings-Hit.espGameFile10=VGreetings-Thief.espGameFile11=VGreetings-Flee.espGameFile12=VGreetings-Idle.espGameFile13=VGreetings-Attack.espGameFile14=VGreetings.espGameFile15=LadyD_Reduced_Commentary.espGameFile16=Texture Fix -TR -1.0.espGameFile17=Better Bodies.espGameFile18=New Argonian Bodies - Clean.espGameFile19=New Khajiit Bodies - Clean.espGameFile20=KS_Ren's_Oblivion_Hair_for_Morrowind_1.0.espGameFile21=WeatheredSigns.espGameFile22=LGNPC_NoLore_v0_83.espGameFile23=Galsiahs Character Development.espGameFile24=GCD StartScript for Trib or Bloodmoon.espGameFile25=GCD better balanced birthsigns.espGameFile26=GCD Restore Potions Fix.espGameFile27=Book Jackets - Morrowind.espGameFile28=Book Jackets - Tribunal.espGameFile29=Book Jackets - Bloodmoon.espGameFile30=KS_Julan_Ashlander Companion_1.3.espGameFile31=Lgnpc_SN.espGameFile32=Wakim's Game Improvement 9.espGameFile33=LGNPC_GnaarMok_v1_10.espGameFile34=Expanded Sounds.espGameFile35=LGNPC_AldVelothi_v1_20.espGameFile36=LGNPC_MaarGan_v1_11.espGameFile37=LGNPC_HlaOad_v1_32.espGameFile38=Vality's Bitter Coast Addon.espGameFile39=LGNPC_Aldruhn_v1_13.espGameFile40=LGNPC_Pelagiad_v1_13.espGameFile41=Better Clothes_v1.1_nac.espGameFile42=LGNPC_TelMora_v1_11.espGameFile43=LGNPC_Khuul_v2_01.espGameFile44=LGNPC_VivecFQ_v2_03.espGameFile45=LGNPC_Indarys_Manor_v1_45.espGameFile46=Less_Generic_Bloodmoon.espGameFile47=LGNPC_Secret_Masters_v1_21.espGameFile48=Vality's Ascadian Isles Addon.espGameFile49=LGNPC_TelUvirith_v1_10.espGameFile50=LGNPC_VivecRedoran_v1_42.espGameFile51=LGNPC_PaxRedoran_v1_12.espGameFile52=LGNPC_SoulSicknessPatch_v1_00.espGameFile53=Bloated Caves.espGameFile54=Graphic Herbalism.espGameFile55=Graphic Herbalism Extra.espGameFile56=Graphic Herbalism ES Patch.espGameFile57=BetterClothes_Patch.espGameFile58=Graphic Herbalism Extra ES Patch.espGameFile59=AIM_MW_TB_BM_1dot0.espGameFile60=Nymerias_Faster_Walk.espGameFile61=Less_Generic_Tribunal.espGameFile62=mel_teleportPlugin_1_1.espGameFile63=mel_teleportPlugin_1_1_Patches.espGameFile64=mel_teleportPlugin_1_1_JulanAddon.espGameFile65=cellnamepatch.espGameFile66=IceNioLivRobeReplacerPLUS.espGameFile67=Graphic Herbalism TR.ESPGameFile68=Piratelords Trade Enhancements.espGameFile69=PTE - BC Patch.espGameFile70=fogpatch.espGameFile71=Mashed Lists.espGameFile72=merged-objects.esp

My merged objects.esp:

http://digi-thal.de/~fallenwizard/temp/merged-objects.esp

» Tue Mar 15, 2011 3:55 am

Hi everyone, thanks for your testing & feedback.

It seems that esper has a lot of problems with certain scripts and enchantments:

Yes, this is basically a problem with failing to update the lengths of string records when the strings themselves changed.
This is now fixed in source. The other problems seen by Dragon32 are also fixed i.e. the failure to treat Morrowind.esm as the source of "original" records, and loss of some subrecords when merging clothing records. The check-resources functionality is also improved -- it *should* work a lot better now though needs testing.

I have added a new option: --find-conflicts. This scans files record-by-record and prints a list of which records are modified by > 1 plugin.

Stay tuned, I will upload a new executable within the next 12 hours.

P.S. someone on the last page said they're going to use this on their game -- I would suggest that's a little brave at this stage, though it's definitely getting closer to usefulness.

» Mon Mar 14, 2011 2:00 pm

That's good news. Esper is getting better and better.

By the way: How can I run esper with a Lisp interpreter? I installed GNU Common Lisp and I can't even start anything with it because it's too complicated.

» Mon Mar 14, 2011 11:45 pm

That's good news. Esper is getting better and better.

By the way: How can I run esper with a Lisp interpreter? I installed GNU Common Lisp and I can't even start anything with it because it's too complicated.

First, delete GNU common lisp, it's ancient. The best common lisp systems are Clozure (*not* the same thing as Clojure, be careful) or SBCL. Esper is developed in clozure.

The best way to get started in CL in windows is probably to download "lisp in a box" at http://gigamonkeys.com/lispbox/lispbox.html. CL's "package-finding" facility (ASDF) is a little complicated to set up, I believe it's already set up in "Lisp in a box". The versions of clozure, SBCL etc it comes with might be a little old.

Expect to spend a while getting things to work. Unfortunately the process of getting CL up and running has a steeper learning curve than python etc.

Basically, once ASDF is working and you can load other systems with it...
You want to put esper source somewhere that ASDF can find it, and type

(asdf:oos 'asdf:load-system :esper)

Then

(in-package :esper)

Then

(main "command line...") to run it
or any other function... the function called esper is the workhorse...

» Tue Mar 15, 2011 3:28 am

Hi guys
New version is up - beta 6.
Don't have time to update the first post just now. The zip file is in the usual place.
Several new features -- see the readme.
Understands {{ESPER:...}} directives in plugin descriptions.
New 'find conflicts' facility.
Enjoy

» Mon Mar 14, 2011 8:29 pm

Thanks

I will try it out.

EDIT:

Something went wrong with Beta 6. The objects esp took forever to merge (up to 5-6 times of Beta 5) is huge (2 MB for just 18 Mods) and the enchantments still don't work.

Unable to locate Enchanting 'dire spark_enes_03'.Unable to locate Enchanting 'dire spark_enir_en'.Unable to locate Enchanting 'dire spark_enastThyEnemy_en'.Unable to locate Enchanting 'dire spark_ene_en'.Unable to locate Enchanting 'cruel flame_enne Close'.Unable to locate Enchanting 'cruel shard_ennation'.Unable to locate Enchanting 'cruel spark_enoration1'.Unable to locate Enchanting 'cruel viper_en_en'.Unable to locate Enchanting 'cruel flame_ennation_en'.Unable to locate Enchanting 'cruel shard_enne Close'.Unable to locate Enchanting 'cruel spark_en3'.Unable to locate Enchanting 'cruel viper_ende_fclothes_01'.

Loaded mods:

[Game Files]GameFile0=Bloodmoon Landscape Overhaul 1.0.esmGameFile1=Bloodmoon.esmGameFile2=Morrowind Patch v1.6.5-BETA.esmGameFile3=Morrowind.esmGameFile4=TR_Data.esmGameFile5=TR_Map1.esmGameFile6=TR_Map2.esmGameFile7=Texture Fix 1.8.esmGameFile8=Tribunal.esmGameFile9=Better Bodies.espGameFile10=Better Clothes_v1.1_nac.espGameFile11=IceBradyHurdyRobeReplacerPLUS.espGameFile12=Mashed Lists.espGameFile13=New Argonian Bodies - Clean.espGameFile14=New Khajiit Bodies - Clean.espGameFile15=TR_CensusAndExciseTravel.espGameFile16=Texture Fix -TR -1.0.espGameFile17=WeatheredSigns.espGameFile18=merged-objects.esp

» Mon Mar 14, 2011 11:06 pm

Good morning.

I am watching this thread with great interest, and hope to see a functional ESPER soon. While I am emphatically NOT a programmer, I do spend a lot of time working with them professionally (I do user support and technical documentation) while helping to develop new programs. I understand just how much time and effort these projects consume, and I applaud the brave souls that take their personal time to do them for the rest of us.

So far, it seems that development on this project is moving along quite rapidly. As with many things, if it were EASY, someone would have done it already. It's very sad that Ghostwheel never saw fit to release the source for TESTOOL, but I'm sure he (or she) had reasons for doing so. Many folks have talked over the years about re-inventing this tool, and it's great to see someone finally stepping up and giving it a shot.

Thanks, and keep up the great work Eeeickythump. You've got a lot of (mostly silent) folks cheering you on!

Rochndil, who will really be able to make ESPER work when he finishes his 200+ mod build...

» Mon Mar 14, 2011 1:58 pm

Thanks, and keep up the great work Eeeickythump. You've got a lot of (mostly silent) folks cheering you on!

Thanks for the kind words!

@FallenWizard:

Regarding the merging process being slower and producing a larger esp: what were your input files? Did they include the Bethesda esms? Thinking about the merging process more, I think it's probably best NOT to include them in the list of files to scan when merging -- I just don't think it's ever necessary. Therefore I think I will prevent merge-records from considering Morrowind/Tribunal/Bloodmoon.esm in future. (If you think I'm wrong feel free to argue.)

So basically, what's happening is that the 'merge records' output plugin includes a lot of unnecessary records because they are modifications of records in the big ESMs. It's slow because it's scanning those big ESMs (assuming you were using them as input). So I *think* esper is "working", just slow because it's scanning those ESMs.

Re the 'unable to locate...' messages - obviously something's not quite right there. What program are they from? I don't think they are produced by esper. Can you tell me what plugins you think those 'enchant' records are found in?

Finally: if you are serious about using esper within a common lisp environment, I have a better idea. I could easily upload a version of esper that enters a lisp interactive environment when run (very similar to the python interpreter if you've ever used that). All you would need to type is (in-package :esper), then you could type names of esper functions to run them, etc etc. Let me know if interested.

» Mon Mar 14, 2011 1:24 pm

So basically, what's happening is that the 'merge records' output plugin includes a lot of unnecessary records because they are modifications of records in the big ESMs. It's slow because it's scanning those big ESMs (assuming you were using them as input). So I *think* esper is "working", just slow because it's scanning those ESMs.

I believe TESTOOL uses a small cache (testool.dat) of just the selected records it needs from the vanilla .esms in order to improve performance. Just an idea, if you wanted to look into it.

» Mon Mar 14, 2011 4:42 pm

Thanks for the kind words!

@FallenWizard:

Regarding the merging process being slower and producing a larger esp: what were your input files? Did they include the Bethesda esms? Thinking about the merging process more, I think it's probably best NOT to include them in the list of files to scan when merging -- I just don't think it's ever necessary. Therefore I think I will prevent merge-records from considering Morrowind/Tribunal/Bloodmoon.esm in future. (If you think I'm wrong feel free to argue.)

So basically, what's happening is that the 'merge records' output plugin includes a lot of unnecessary records because they are modifications of records in the big ESMs. It's slow because it's scanning those big ESMs (assuming you were using them as input). So I *think* esper is "working", just slow because it's scanning those ESMs.

Re the 'unable to locate...' messages - obviously something's not quite right there. What program are they from? I don't think they are produced by esper. Can you tell me what plugins you think those 'enchant' records are found in?

I used "esper -r -a -o merged-objects.esp", so I think it'll include all esms. Maybe the enchantments are messed up because the esp include the esm files. I will try to tag them with {{ESPER: NOMERGE}} later.

Finally: if you are serious about using esper within a common lisp environment, I have a better idea. I could easily upload a version of esper that enters a lisp interactive environment when run (very similar to the python interpreter if you've ever used that). All you would need to type is (in-package :esper), then you could type names of esper functions to run them, etc etc. Let me know if interested.

No, you don't need to do it. I dropped the idea of running it in LISP. It's just too complicated.

» Tue Mar 15, 2011 3:32 am

No, you don't need to do it. I dropped the idea of running it in LISP. It's just too complicated.

OK. But just to be clear -- you wouldn't need to download or set up anything. The reason esper.exe is 26MB is that it is actually an entire, self-sufficient common lisp compiler and interpreter. Giving the user access to that interpreter just requires me compiling esper slightly differently.

So the only complicated thing would be learning how to interact with the lisp system. Anyway let me know if you change your mind.

» Tue Mar 15, 2011 12:31 am

-snip- ... Therefore I think I will prevent merge-records from considering Morrowind/Tribunal/Bloodmoon.esm in future. (If you think I'm wrong feel free to argue.) ... -snip-

How would that affect merging two or more plugins that all modify a stock object? From what Dragon32 had tested, just merging the two plugins that intended to modify the stock gloves did not add their changes to the stock object but instead the second overrode the first. Your suggestion was to include a third base plugin - in Dragon32's case it would need to be morrowind.esm.

Like john.moonsugar stated, the method TESTool uses is to cache the stock esm data on startup to give a baseline - maybe include a command line option for using the stock esms as the original object sources that the other plugins will be compared against, and compare against a data cache to help speed up the processing.

Finally: if you are serious about using esper within a common lisp environment, I have a better idea. I could easily upload a version of esper that enters a lisp interactive environment when run (very similar to the python interpreter if you've ever used that). All you would need to type is (in-package :esper), then you could type names of esper functions to run them, etc etc. Let me know if interested.

Although I probably wouldn't have interest in using such a version... couldn't hurt to do so, especially if it results in a significant reduction in download size. I personally have no problems dl'ing large files, but others might benefit from a smaller dl. On the other hand, someone who hasn't the patience to wait for a large dl might not have the patience to learn how to interact with the lisp system. <_<

» Tue Mar 15, 2011 3:59 am

How would that affect merging two or more plugins that all modify a stock object? From what Dragon32 had tested, just merging the two plugins that intended to modify the stock gloves did not add their changes to the stock object but instead the second overrode the first. Your suggestion was to include a third base plugin - in Dragon32's case it would need to be morrowind.esm.

...Actually, thinking more about it, I think you're right. It does need to include the besthesda ESMs.

Like john.moonsugar stated, the method TESTool uses is to cache the stock esm data on startup to give a baseline - maybe include a command line option for using the stock esms as the original object sources that the other plugins will be compared against, and compare against a data cache to help speed up the processing.

Yes, I probably will do this at some point. Currently though, "correctness" is a higher priority than "speed". I would like everything to be working before I complicate the program logic further with something like this. Beware premature optimization.

» Mon Mar 14, 2011 4:34 pm

The technical nature of a lot of this is, surely, beyond me -- but nonetheless thank you for all the work that's been done so far! The thought that a mature Esper program could fix yet another string of issues is awesome...

I bow before the Morrowind community. I consider Morrowind to be my first "real" game and have enjoyed it for so very long, now. It has always had flaws. But with every month, every year that passes by into the distance... the limitless ingenuity of the Morrowind community polishes more of the game. Makes it more complete. More enjoyable. More awe-inspiring.

Needless to say I'll be keeping an eye on the thread =)

» Tue Mar 15, 2011 3:16 am

...
Like john.moonsugar stated, the method TESTool uses is to cache the stock esm data on startup to give a baseline - maybe include a command line option for using the stock esms as the original object sources that the other plugins will be compared against, and compare against a data cache to help speed up the processing.

Yes, I probably will do this at some point. Currently though, "correctness" is a higher priority than "speed". I would like everything to be working before I complicate the program logic further with something like this. Beware premature optimization.

Are we talking about second or minutes of 'speed' difference when processing the ESM files or not. In my small tests scenarios with a new game and some test mods plus some save games I don't have problems with the speed of this program. But I'll admit that I'm a patient guy

. I'm aware that it's still a beta version, but I have a comprehensive knowledge about TESTool from a user perspective and with my small experience as a modder. Keep up the good work.