esByteTypes.dll esTypeLibrary.dll : III

» Mon May 07, 2012 3:05 pm

Hey All,

First post is a question I'm not expecting an answer to. I've been playing around with Morrowind ESM/ESP files and while I've mostly been successful I've run into a few problems mapping certain data types in an ESM with the data types used by languages other than C. Specifically the 4-byte "short" stored in some GLOB records, and anything with a Long64.

I noticed these 2 DLLs in my Morrowind folder. esTypeLibrary.dll and esByteTypes.dll. I'm not sure but I suspect they were installed with the Enchanted editor. Does anyone know who wrote them? Is there source available? If they are useful then I'd be very happy to import the functions and use them but I'd need the .h or a list of functions and what they take as parameters.

x24

» Mon May 07, 2012 11:59 am

They are definitely from Enchanted Editor. I don't know much about where EE came from, but I remember hearing someone say that the developer hasn't been heard from for a long time... hopefully someone else can tell you more.

» Mon May 07, 2012 7:45 pm

They are definitely from Enchanted Editor. I don't know much about where EE came from, but I remember hearing someone say that the developer hasn't been heard from for a long time... hopefully someone else can tell you more.

Thanks. I've solved the problem of reading a 4 byte short. I'm using http://www.mwmythicmods.com/argent/tech/es_format.html to build a DTD and dump to XML. Maybe eventually from XML back to binary. I still can't read the long64 file size in the MAST/DATA records, but HEDR, GLOB, and GMST records are coming in fine.

» Tue May 08, 2012 1:24 am

Thanks. I've solved the problem of reading a 4 byte short. I'm using http://www.mwmythicmods.com/argent/tech/es_format.html to build a DTD and dump to XML. Maybe eventually from XML back to binary. I still can't read the long64 file size in the MAST/DATA records, but HEDR, GLOB, and GMST records are coming in fine.

Interesting, I've just started work to do similarly with SmartMerger, although the progress may be slow, not because it is hard but I am picking up enough information that I may try to convert to C++.

If you have any good ideas for the XML structure, let me know.

As for the long64, it's little endian, and 99.99999% of the time you could hack it and only read the first 32bit to get the correct data. (Least I think that's right), I highly doubt ANY plugin or master will be over 4 Gigs

» Mon May 07, 2012 10:32 pm

Expand on can't read. The data is wrong?

The data is as you would expect, little endian twos complement integers.

» Mon May 07, 2012 3:58 pm

Yes, can't read meant the data was wrong. Reading only the first 4 bytes as a long worked.

I'm not a C programmer. I'm a documents and database guy. Lotus Notes/Domino/XPages and various relational database and pretty much anything derived from SGML like HTML, XML etc.

A published DTD would be immensely useful. There are just so many tools out there for doing cool things with XML. Since niftoaster and PyFFI require Python, I expect that a python script which converts back and forth from ES? to XML would be the most useful to the greatest number of people. In which case, I'm using the wrong language.

» Mon May 07, 2012 5:09 pm

Reading only the first 4 bytes as a long worked.

Yay to partial hacks... Consider that a workaround until you can actually read it as a long64. (Although that's the only place you use a 64bit value anywhere in the file structure)

I don't think anyone's really tried to work on converting the data to and from XML for the TES3 format. I've just begun work, likely won't be making huge progress but getting the functionality won't take too long. As for a DTD, not sure on that. I know the basic structure and use of an XML, but not all the details of how they are fully used.

» Mon May 07, 2012 1:28 pm

Heh. XML and I don't get on. But there are a nice amount of tools.

Regarding your problems, I just checked a fairly reliable code base (Wrye Mash, not that you even want to look at that code base imho :sick:

) and it like the file format you linked to is correct. (Nb: Mash it is unsigned). I am happy to look at your code as long as it isn't in something to esoteric.

Era Scarecrow: Peachykeen did something with xml conversion, although I am not sure he got anywhere. Probably on his github account.

» Mon May 07, 2012 7:11 pm

This is written in an old VB clone published by IBM. If there are functions and objects I can pull out of morrowind.exe or the DLLs I'd rather do that.
They are far more likely to read and write the correct data in the correct format than I am.

From the Docs:

On Windows importing DLL functions is done using the STDCALL calling convention.
If you are using a C++ compiler, the name of any function becomes corrupted. Use the extern "C" {. . .} construct to keep the exact name intact.
If you are using Windows 95 or Windows NT, the name of an exported DLL function is case sensitive but automatically converted to uppercase in the Declare statement. To successfully call an exported DLL, use the Alias clause in the Declare statement to specify the function name with correct capitalization. The alias is left alone.

Example:

' The following statements declare an exported DLL with an ''alias (preserving case sensitivity), and then call that ''function.'Declare Function DirDLL Lib "C:\myxports.dll" _   Alias "_HelpOut" (I1 As Long, I2 As Long)DirDLL(5, 10)

Option PublicOption DeclareUse "crjc"'// Putting these read ops in functions stops them from moving the''// internal file pointer in an object that uses them.'Function GetShort4Value(file As Integer, filePos As Long) As Integer	'// Gets a 2 byte short from a 4 byte field in an ES3 file.'	Dim subfloat As Single	Get #file, filePos, subfloat	If subfloat > 32768 Then subfloat = 0	If subfloat < (-32768) Then subfloat = 0	GetShort4Value = http://forums.bethsoft.com/topic/1316957-esbytetypesdll-estypelibrarydll/Cint(Fix(subfloat))End FunctionFunction GetByteValueAsString(file As Integer, filePos As Long) As String'// Returns a 1 byte char field in a binary file as a String'	Seek #file, filePos	GetByteValueAsString = Cstr(Input(1, #file))End FunctionFunction GetByteValueAsInt(file As Integer, filePos As Long) As Integer	'// Returns a 1 byte char field in a binary file as an Integer'	Dim getbyte As Byte	Get #file, filePos, getbyte	GetByteValueAsInt = Cint(getbyte)End FunctionFunction GetShortValue(file As Integer, filePos As Long) As Integer	'// Returns a 2 byte field in a binary file as an Integer'	Get #file, filePos, GetShortValueEnd FunctionFunction GetDoubleValue(file As Integer, filePos As Long) As Double	'// Returns an 8 byte field in a binary file as a signed Double float'	Get #file, filePos, GetDoubleValueEnd FunctionFunction GetFloatValue(file As Integer, filePos As Long) As Single	'// Returns an 4 byte field in a binary file as a Single'	Get #file, filePos, GetFloatValueEnd FunctionFunction GetLongValue(file As Integer, filePos As Long) As Long	'// Returns an 4 byte field in a binary file as a signed 4 byte integer'	Get #file, filePos, GetLongValueEnd FunctionFunction GetStringValue(file As Integer, filePos As Long, slength As Integer) As String	'// Returns chars in field in a binary file as a String'	Seek #file, filePos	GetStringValue = http://forums.bethsoft.com/topic/1316957-esbytetypesdll-estypelibrarydll/Input(slength, #file)End FunctionFunction GetRecordName(file As Integer, filePos As Long) As String'// Returns the record names used in an ES3 file as a String'	Seek #file, filePos	GetRecordName = Input(4, #file)End Function'// ******************************************************************''// An object with the contents of a HEDR subrecord in an ES3 file''// This is the file header. There is only one per file.'Class TES3_HEDR		'4 bytes, float Version (1.2)'	Public version As Single		'4 bytes, file type (0=esp file; 1=esm file; 32=ess file)'	Public fileType As Long		'32 Bytes, Company Name string'	Public companyName As String		'256 Bytes, ESM file description?'	Public Description As String		'4 bytes, long NumRecords (48227)'	Public numRecords As Long	Sub New(file As Integer, pos As Long)		version = GetFloatValue(file, pos)		pos = pos + 4		fileType = GetLongValue(file, pos)		pos = pos + 4		companyName = GetStringValue(file, pos, 32)		pos = pos + 32		Description = GetStringValue(file, pos, 256)		pos = pos + 256		numRecords = GetLongValue(file, pos)		pos = pos + 4	End SubEnd Class'// ******************************************************************''// An object with the contents of a SKDT subrecord in an ES3 file''// This is skill data from a SKIL record.'Class TES3_SKDT		'long Attribute'	Public Attribute As Long		'long Specialization'		'	0 = Combat'		'	1 = Magic'		'	2 = Stealth'	Public Specialization As Long		'float UseValue[4]'		'The use types for each skill are hard-coded.'	Public UseValue1 As Single	Public UseValue2 As Single	Public UseValue3 As Single	Public UseValue4 As Single	Sub New(file As Integer, pos As Long)		Attribute = GetLongValue(file, pos)		pos = pos + 4		Specialization = GetLongValue(file, pos)		pos = pos + 4		UseValue1 = GetFloatValue(file, pos)		pos = pos + 4		UseValue2 = GetFloatValue(file, pos)		pos = pos + 4		UseValue3 = GetFloatValue(file, pos)		pos = pos + 4		UseValue4 = GetFloatValue(file, pos)		pos = pos + 4	End SubEnd Class'// ******************************************************************''// An object with the contents of a CLDT subrecord in an ES3 file''// This is class data from a CLAS record.'Class TES3_CLDT		'long AttributeID1'	Public AttributeID1 As Long		'long AttributeID2'	Public AttributeID2 As Long		'long Specialization?'		'	0 = Combat'		'	1 = Magic'		'	2 = Stealth'	Public Specialization As Long		'long MinorID1'	Public MinorID1 As Long		'long MajorID1'	Public MajorID1 As Long		'long MinorID2'	Public MinorID2 As Long		'long MajorID2'	Public MajorID2 As Long		'long MinorID3'	Public MinorID3 As Long		'long MajorID3'	Public MajorID3 As Long		'long MinorID4'	Public MinorID4 As Long		'long MajorID4'	Public MajorID4 As Long		'long MinorID5'	Public MinorID5 As Long		'long MajorID5'	Public MajorID5 As Long		'long Flags'		'	0x0001 = Playable'	Public Flags As Long		'long AutoCalcFlags'		'	0x00001 = Weapon'		'	0x00002 = Armor'		'	0x00004 = Clothing'		'	0x00008 = Books'		'	0x00010 = Ingrediant'		'	0x00020 = Picks'		'	0x00040 = Probes'		'	0x00080 = Lights'		'	0x00100 = Apparatus'		'	0x00200 = Repair'		'	0x00400 = Misc'		'	0x00800 = Spells'		'	0x01000 = Magic Items'		'	0x02000 = Potions'		'	0x04000 = Training'		'	0x08000 = Spellmaking'		'	0x10000 = Enchanting'		'	0x20000 = Repair Item'	Public AutoCalCFlags As Long	Sub New(file As Integer, pos As Long)		AttributeID1 = GetLongValue(file, pos)		pos = pos + 4		AttributeID2 = GetLongValue(file, pos)		pos = pos + 4		Specialization = GetLongValue(file, pos)		pos = pos + 4		MinorID1 = GetLongValue(file, pos)		pos = pos + 4		MajorID1 = GetLongValue(file, pos)		pos = pos + 4		MinorID2 = GetLongValue(file, pos)		pos = pos + 4		MajorID2 = GetLongValue(file, pos)		pos = pos + 4		MinorID3 = GetLongValue(file, pos)		pos = pos + 4		MajorID3 = GetLongValue(file, pos)		pos = pos + 4		MinorID4 = GetLongValue(file, pos)		pos = pos + 4		MajorID4 = GetLongValue(file, pos)		pos = pos + 4		MinorID5 = GetLongValue(file, pos)		pos = pos + 4		MajorID5 = GetLongValue(file, pos)		pos = pos + 4		Flags = GetLongValue(file, pos)		pos = pos + 4		AutoCalCFlags = GetLongValue(file, pos)		pos = pos + 4	End SubEnd Class'// ******************************************************************''// An object with the contents of a FADT subrecord in an ES3 file''// This is faction data from a FACT record.'Class TES3_FADT		'FADT = Faction data (240 bytes)'		'	long AttributeID1'	Public AttributeID1 As Long		'	long AttributeID2'	Public AttributeID2 As Long		'	RankData[10]'		'		long Attribute1'		'		long Attribute2'		'		long FirstSkill'		'		long SecondSkill'		'		long Faction'	Public R1Attribute1 As Long	Public R1Attribute2 As Long	Public R1FirstSkill As Long	Public R1SecondSkill As Long	Public R1Faction As Long	Public R2Attribute1 As Long	Public R2Attribute2 As Long	Public R2FirstSkill As Long	Public R2SecondSkill As Long	Public R2Faction As Long	Public R3Attribute1 As Long	Public R3Attribute2 As Long	Public R3FirstSkill As Long	Public R3SecondSkill As Long	Public R3Faction As Long	Public R4Attribute1 As Long	Public R4Attribute2 As Long	Public R4FirstSkill As Long	Public R4SecondSkill As Long	Public R4Faction As Long	Public R5Attribute1 As Long	Public R5Attribute2 As Long	Public R5FirstSkill As Long	Public R5SecondSkill As Long	Public R5Faction As Long	Public R6Attribute1 As Long	Public R6Attribute2 As Long	Public R6FirstSkill As Long	Public R6SecondSkill As Long	Public R6Faction As Long	Public R7Attribute1 As Long	Public R7Attribute2 As Long	Public R7FirstSkill As Long	Public R7SecondSkill As Long	Public R7Faction As Long	Public R8Attribute1 As Long	Public R8Attribute2 As Long	Public R8FirstSkill As Long	Public R8SecondSkill As Long	Public R8Faction As Long	Public R9Attribute1 As Long	Public R9Attribute2 As Long	Public R9FirstSkill As Long	Public R9SecondSkill As Long	Public R9Faction As Long	Public R10Attribute1 As Long	Public R10Attribute2 As Long	Public R10FirstSkill As Long	Public R10SecondSkill As Long	Public R10Faction As Long		'	long SkillID[6]'	Public SkillID1 As Long	Public SkillID2 As Long	Public SkillID3 As Long	Public SkillID4 As Long	Public SkillID5 As Long	Public SkillID6 As Long		'	long Unknown1 (-1)?'	Public Unknown1 As Long		'	long Flags'		'		1 = Hidden from Player'		Public Flags As Long	Sub New(file As Integer, pos As Long)		AttributeID1 = GetLongValue(file, pos)		pos = pos + 4		AttributeID2 = GetLongValue(file, pos)		pos = pos + 4		R1Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R1Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R1FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R1SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R1Faction = GetLongValue(file, pos)		pos = pos + 4		R2Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R2Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R2FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R2SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R2Faction = GetLongValue(file, pos)		pos = pos + 4		R3Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R3Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R3FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R3SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R3Faction = GetLongValue(file, pos)		pos = pos + 4		R4Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R4Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R4FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R4SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R4Faction = GetLongValue(file, pos)		pos = pos + 4		R5Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R5Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R5FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R5SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R5Faction = GetLongValue(file, pos)		pos = pos + 4		R6Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R6Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R6FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R6SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R6Faction = GetLongValue(file, pos)		pos = pos + 4		R7Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R7Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R7FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R7SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R7Faction = GetLongValue(file, pos)		pos = pos + 4		R8Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R8Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R8FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R8SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R8Faction = GetLongValue(file, pos)		pos = pos + 4		R9Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R9Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R9FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R9SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R9Faction = GetLongValue(file, pos)		pos = pos + 4		R10Attribute1 = GetLongValue(file, pos)		pos = pos + 4		R10Attribute2 = GetLongValue(file, pos)		pos = pos + 4		R10FirstSkill = GetLongValue(file, pos)		pos = pos + 4		R10SecondSkill = GetLongValue(file, pos)		pos = pos + 4		R10Faction = GetLongValue(file, pos)		pos = pos + 4		SkillID1 = GetLongValue(file, pos)		pos = pos + 4		SkillID2 = GetLongValue(file, pos)		pos = pos + 4		SkillID3 = GetLongValue(file, pos)		pos = pos + 4		SkillID4 = GetLongValue(file, pos)		pos = pos + 4		SkillID5 = GetLongValue(file, pos)		pos = pos + 4		SkillID6 = GetLongValue(file, pos)		pos = pos + 4		Unknown1  = GetLongValue(file, pos)		pos = pos + 4		Flags  = GetLongValue(file, pos)		pos = pos + 4	End SubEnd Class'// ******************************************************************''// A subrecord in an ES3 file'Class TES3Subrecord	Public RecordName As String	Public RecordLength As Long	Public RecordStart As Long	recordPos As Long	recordEnd As Long	file As Integer	Sub New(f As Integer, p As Long)		file = f		RecordStart = p		recordPos = p		RecordName = GetRecordName(file,recordPos)		recordPos = recordPos + 4		RecordLength = GetLongValue(file,recordPos)		recordPos = recordPos + 4	End Sub	Function GetString() As String ' Variable length chars'		GetString = GetStringValue(file,recordPos,Cint(RecordLength))	End Function	Function GetLong() As Long ' 4 byte signed integer'		GetLong = GetLongValue(file,recordPos)	End Function	Function GetFloat() As Single '4 byte floating point'		GetFloat = GetFloatValue(file,recordPos)	End Function	Function GetDouble() As Double ' 8 byte floating point'		GetDouble = GetDoubleValue(file,recordPos)	End Function	Function GetShort() As Integer ' 2 byte signed integer'		GetShort = GetShortValue(file,recordPos)	End Function	Function GetShort4() As Integer ' 2 byte signed integer from 4 bytes'		GetShort4 = GetShort4Value(file,recordPos)	End Function	Function GetByteAsInt() As Integer ' 2 byte unsigned int from 1 byte'		GetByteAsInt = GetByteValueAsInt(file,recordPos)	End Function	Function GetByteAsString() As String ' 1 char'		GetByteAsString = GetByteValueAsString(file,recordPos)	End Function	Function GetHEDR() As TES3_HEDR		If Me.RecordName = "HEDR" Then			Set GetHEDR = New TES3_HEDR(file,recordPos)		End If	End Function	Function GetCLDT() As TES3_CLDT		If Me.RecordName = "CLDT" Then			Set GetCLDT = New TES3_CLDT(file,recordPos)		End If	End Function	Function GetFADT() As TES3_FADT		If Me.RecordName = "FADT" Then			Set GetFADT = New TES3_FADT(file,recordPos)		End If	End Function	Function GetSKDT() As TES3_SKDT		If Me.RecordName = "SKDT" Then			Set GetSKDT = New TES3_SKDT(file,recordPos)		End If	End FunctionEnd Class'// ******************************************************************''// A record in an ES3 file'Class TES3Record	Public RecordName As String	Public RecordLength As Long	Public RecordHeader As Long	Public RecordFlags As Long	Public MoreSubs As Boolean	recordPos As Long	Public RecordStart As Long	Public RecordEnd As Long	subFirst As Long	subNext As Long	file As Integer	Sub New(f As Integer, rs As Long)		RecordStart = rs		recordPos = rs		file = f		RecordName = GetRecordName(file,recordPos)		recordPos = recordPos + 4		RecordLength = GetLongValue(file,recordPos)		recordPos = recordPos + 4		RecordHeader = GetLongValue(file,recordPos)		recordPos = recordPos + 4		RecordFlags = GetLongValue(file,recordPos)		recordPos = recordPos + 4		subFirst = recordPos		subNext = recordPos		RecordEnd = recordStart + RecordLength + 16		MoreSubs = True	End Sub	Function GetFirstSubrecord() As TES3Subrecord		Set GetFirstSubrecord = New TES3Subrecord(file, subFirst)		subNext = subFirst + GetFirstSubrecord.RecordLength + 8		If subNext >= RecordEnd Then			MoreSubs = False		End If	End Function	Function GetNextSubrecord() As TES3Subrecord		If MoreSubs Then			Set GetNextSubrecord = New TES3Subrecord(file, subNext)			subNext = subNext + GetNextSubrecord.RecordLength + 8			If subNext >= RecordEnd Then				MoreSubs = False			End If		End If	End FunctionEnd Class'// ******************************************************************''// An ES3 file'	Class TES3File	Public FileLength As Long	Public FilePath As String	Public FileName As String	Public Version As Single	Public FileType As Long	Public CompanyName As String	Public Description As String	Public NumRecords As Long	Public MoreRecords As Boolean	file As Integer	filePos As Long	recordStart As Long	recordLength As Long	recordPrevious As Long	recordNext As Long	recordNumber As Long	Sub New(f As String)		' Where f is the full path and file name'		FilePath = f		FileName = GetFileNameFromPath(FilePath)		file = Freefile()		Open FilePath For Binary Access Read As file		FileLength = Lof(file)		filePos = 1		Dim r As TES3Record		Dim s As TES3Subrecord		Dim h As TES3_HEDR		MoreRecords = True		Set r = Me.GetFirstRecord()		If r.RecordName = "TES3" Then			Set s = r.GetFirstsubRecord()			If s.RecordName = "HEDR" Then				Set h = s.GetHEDR()				Version = h.Version				FileType = h.FileType				CompanyName = h.CompanyName				Description = h.Description				NumRecords = h.NumRecords			End If		End If	End Sub	Sub Delete()		Close #file	End Sub	Function GetFirstRecord() As TES3Record		Dim recordPos As Long		filePos = 1		recordPos = 1		recordPrevious = 1		recordStart = 1		recordNumber = 1		Set GetFirstRecord = New TES3Record(file, recordPos)		recordNext =  + GetFirstRecord.recordLength + 17		If recordNext >= FileLength Then MoreRecords = False	End Function	Function GetNextRecord() As TES3Record		Dim recordPos As Long		recordPrevious = recordStart		recordStart = recordNext		recordPos = recordNext		If recordPos >= FileLength Then			Exit Function		End If		Set GetNextRecord = New TES3Record(file, recordPos)		recordNext = recordStart + GetNextRecord.recordLength + 16		If recordNext >= FileLength Then MoreRecords = False	End FunctionEnd Class