Ah ok. I get you now. Yeah, good idea. It would be unecessary at the data gathering end.

True, but collecting and storing the data locally with MWSQL then makes it readily available to pass back to other MWSE mods written to specifically make use of the data without having to connect with a remote db.

Well, on this morrowficated reproduction of WoW Armory? I suggest the best would be to start at the beginning.

I have taken a short peak on the application and it looks promising. In fact, it's far better then I first hoped. And if we just get this project kick-started then we can run it smoothly to shore. But, at the moment, I can't and shouldn't give any assumptions or summaries how everything will turn out. Further more, will do some more read-up on database management this weekend to refresh the 'ol knowledge-bank.

Would also be great if you could answer to my email.

Cheerio.

I understand your point Yacoby, and it does make perfect sense.

The other side of the argument though is that there can be other uses for the stored data. I know of some people who play games with a cold boot, often turning off network devices as well. I think it would be neat to have the option to upload the data at a later point in time, rather than just when playing. Then as tetchy said, it can be used for other mod purposes.

To be honest, the real reason I first though to use MWSQL is because that's how I got the idea. I was just thinking of the possibilities that it presented, and that's where my train of thought went. If there is a better way to do it, without sacrificing the functionality that I would prefer to have, I'm all for it.


You emailed me? Or are you saying that you prefer if we would discuss via email?

I did sent you an email, but it reason were none of private matter so I might as well spill it out in this thread.

There is a thing I'm wondering about in this soup, and that's question is how does one use this program. I tried first to put the folder in Morrowind, start the "mwsql" application and start the game, Nothing. I tried reading manual inside the folder, and boy, to my knowledge it could really benefit from some re-writting. I read the entire thing, still no clue how to proceed. It might that I'm missing something, if it's in the program or the text-files I don't know. But I know that a little guidelines onto how to operate this mod, would be most appreciated.

Cheerio.

Rather then having it constantly on, I prefer the idea of it being local so I can choose when I want my character info to be uploaded. Not to mention, with my idea of being able to face off against other people's characters in an arena-like situation being kept in mind, if done locally it could be extended even further.

Just an Idea: What about using XML as the format the characters are stored in? Then the files could be uploaded and displayed very easily.
Of course, this would probably require a separate program which parses XML to be usable by MWSE.

Could do db design if you need me to. More used to MS Sql and Oracle but MySql not a problem.


Forgive shyness but it's only my second time posting and not used to posting editor

You would need db columns sort of like the following:-


Internal ID Assigned by back end db
Fliggs ID Assigned by Fliggerty Registration
UserName Username (Beth ID?)
Character Name Name of User Character
Email ?Not Sure if needed
Attribs Character Attributes
Stats Character Stats
Race+Class+Sign So as you could search by preference
Location Currect Location
Current Mods ?Would need to parse Morrowind.ini
Photo What Charcater Looks like

I'm not much of a modder and it's been ages since I worked on client-side apps, but my two cents:

Whether you store the data in files or in a local db (which is stored in files too), the transfer would still work the same way in both cases.

From what I understand, it's one user's data that has to be stored temporarily and then the latest entry(ies) is sent to a server so everybody can access it, like say, on a webpage. Meaning you probably just need one table row per character, which isn't a lot. Even someone completely crazy shouldn't have thousands of different characters, and even someone that does wouldn't play with all of them on a regular basis (meaning the latest stats wouldn't change).

If you needed to do the opposite , meaning all of the server' stored data sent to each users, so your program can access it without an internet connection and without busting your server (for exemple, if you wanted to make the whole content of uesp.net available in-game...), using a local db would be a good option then.

In your case, you can probably just output the necessary info to a file, then have a program that grabs the last valid line(s), packs it as much as possible, does some basic encoding, posts it to your server, which then decodes it, parses it, checks for integrity and stores it in its own db. Every once in a while, your program can also rotate the local output file so it doesn't get too long. You can just dump it or archive it somewhere, thought from my understanding, I don't think you have much use for what were a character's stats at every 5 minutes interval for the past 3 months.

Well I think this is a very cool idea
I'd love to be involved in this. The way I see it, sending game data from a player computer to the db server is np whatsoever. The hard part imo is to make a cool website which displays this game data in a cool way (like so often in software design, the user interface is the hardest part).
But.. this is (can be) only the beginning.. I can imagine loads of cool things that can be done with this..especially when combining Fliggerty's original idea of using the local mwsql db.

Hm. Competition ahead? Or.. as Moonbeam I agree there is a necessarily for columns to make this happen, and also the usage of tables might now be a dumb idea. I getting more and more fond in this project, and would love to see it realized. But despite that, if this project is going to have all those talked about, and suggest features, (arena fighting other characters, enable to view material and all kind of inventory, perhaps even throw in a way to use the stats on a companion, that would enable I think as much multi-player as possible) well, then it blows up the proportions.

Therefore, with all stuff needed to be done, another collage to work with sounds like a perfect solution from my side.
I hope both moonbeam and Fliggerty can agree with me on that. It's just a mere suggestion after all.


Agrees.


Couldn't agree more. Offline services, sounds and is great.



Ages huh? Anyhow, that's some real quality information, appreciate it. Using update functions to rotate the data back and forth between the server and local server is great idea. Or have a way for downloading it manually. I assume that Fliggery is not thinking it to become a automatic station for sending data, instead more in a community perspective if I'm correct. With images, personal information, char. stats, that bit. But, as I said, I'm not sure on that point.

I wonder if we should use Java-coding into this also. SQL with a routing to a Java program could work. But, while java provides more functional and different ways of building the database, it also is makes it more troubling to maintain it. Hm, I will think about that more.

Nonetheless, love to as much help and friends aboard with this thingie. Cause, the more the merrier I say. (man that sounded gay)

Anyway, getting late now so i'm off to bed, Nighty all.

Few things:
- MWSQL uses SQLite, not MySQL (or MSSQL or anything else). SQLite has pretty good docs.
- Still dunno what's up with my manual. Looks decent. Any detailed questions can be directed to me.
- If it helps, I'm going to put the MWSQL source on Google code (as soon as I remember to). It's written in standard C++
- It may be possible to allow it to connect directly to the database.
- However, that takes out another layer of encryption.
- If, when the app to do all this is complete, it has secure data, I can give it a once-over to check and see if I can encrypt it a little for it. It's never truly secure, unless you really mix it up in the code.
- MWSQL actually works?
- SQLite uses a universal-ish data format for columns, so type (and size) don't matter.

@ Rananka: Making the site is no problem. PHP has a very nice working MySQL module.

@ verm1n: Any questions on MWSQL...



It might be better to have MWSQL connect to the online db, and on save (I'm working on a per-user/mygames app, which would wrap/intercept the open file calls from MW) upload the data. So, every time you save your game, it ups. It would save server load, especially given how many people play Morrowind. Also, it might be useful to implement SQLite's transaction feature, to make sure data transfer completes before it's sent.

As I mentioned before, I wanted to use MWSQL instead of just files so that we don't have to write everything each time it updates. Performance is always a concern, especially when we deal with some of MWSE's functions. If we use a file, it becomes necessary to write everything to the file again when we update something.

For example, say we write four string variables to a file in order:

string1
string2
string3
string4

Later on it becomes necessary to update the contents of string2. Because MWSE truncates the file after writing to it, string3 and string4 will be lost and will have to be written again. But with SQL, obviously we can write one location and leave the rest alone. I think that this will put far less load on the scripting side of things. There will be some things that will only need to be written once, or only rarely, while others will change everytime the update runs.

I strongly prefer to give people the option to have it update in real-time, or to simply just upload it at their leisure.

I do also agree that only one line will be needed per character. I think that this database will be rather complex when we get done, but no more so than a decent forum. Every user will be able to have as many characters as they choose.

I'm glad that there is a lot of excitement for this from the creative spectrum, rather than just a lot of people saying "good I can't until you finish it!" That's quite a bit different from what I'm used to. I will gladly welcome all of the help I can get. I love that we already seem to have a variety of talent and knowledge offering opinions. Shared workload is a wonderful thing. I have very little experiencing building anything with PHP + MySQL from scratch, most of my experience there has been modifying what is freely available. I have looked at this as a good opportunity to learn. So I ask that any who wish to be involved to understand that and bear with me; I am open to all suggestions, but in some cases I might need some extra time and hand-holding. As for the scripting side of it, making the actual esp, I think it's safe to say that I can easily handle that.

So, shall we begin? Would a private forum where those who wish to contribute be helpful? Or shall we continue this here in public threads? There's advantages to both sides I think, but I'd like to choose one and stick with it.

Then we need to identify the exact steps that need to be taken to completion, and make "assignments." I may not be a "real" programmer, but I sure can manage a group of people well. :hehe:


Edit: peachykeen, could it possible to have a gui for MWSQL that would let you choose between writing to the online database and writing to a local one? Then the player could decide what to do, and we can easily have the best of both worlds.

Um... I could maybe do something like MGE's video texture, with the # prefix. Applied to the -open command. I haven't worked with any internet code in C++, however (I can open local files no problem ). This was basically the app I learned medium-level cpp on, and so suffered.


I may be able to throw together a simple lister for the database, so long as you can get it online in a SQLite (or possible MySQL, although there are a few differences) database. That I know how to do (wrote a simple image gallery, per user images, titles, descriptions).
Using a UID, then character name to identify each row, it would be easy to have a few dozen columns for various things. I'm not sure how inventory would be handled, but stats are easy...

A simple way to handle replication of data you collect in a peripheral database to a central server is to use the database's built-in logging mechanism. This is very easy to do with MySQL. It avoids the complications that come from having different representations of data on the player's computer and the central server, and it solves the problem Fliggerty was concerned about: how to avoid rewriting a local file for single value changes.

Assume you have a local database (if your central database is MySQL, best if the local one is MySQL; that way, the logs are compatible without further effort) on the player's computer, and your big central database.

As the local database records data, it produces statements that insert or update data. These statements get captured in a log and can be replayed. (There are good reasons for doing so; for example, this is how you recover a database after a catastrophic failure: go back to the last known good database, and replay all the statements since.)

To send an update to your central server, the local database can just pack up its log and send it.

Your central server can replay the log against its database, and that captures the player's local data without further complication.

You still need some logic that resides on the player's computer to know when to cut off the current log, start a new one, and send all the logs the server hasn't acknowledged.

You still need some logic on the server to acknowledge logs that have been received and replayed, and to tell players' computers to stop trying to send logs that have already been received or that are unreadable.

The simplest way to transmit the logs is HTTP (could be HTTPS with TLS or SSL). You can use "cURL" for the HTTP client (which runs fine on Windows) and Apache for the HTTP server. These also have all the authentication features you would need to prevent malefactors from spoofing or hacking your database.

You can use MySQL with its lightweight MyISAM engine on the player's computer, unless you need to organize your data collection into transactions. If you must have transactions, you can use the InnoDB engine.

Fantastic info, I really think I have a good jumping off place. Thank you sincerely.

I think that when we come up with a good name for this project I'll start a proper WIP thread. Off the top of my head: TES III Character Showcase. Other suggestions?

I just finished building a LAMP server today. We'll use it for a build platform.

How about: "The Morrowind Census and Excise Office"?

Or something along those lines at least.

Edit: Sorry fellows, I don't know if can continue on this project. Because it clearly seems that all information is embedded in only experience-knowledge. Whenever it's the functions, the application or other parts in mwse and mwsql I can't really get my head around it. I have researched for information about MWSE and MGE and found out this is not a new thing. Some threads on this forums goes several years back.. the time when MWSE was invented, and you guys, Fliggerty, peachykeen was active in it's development. Being a new-comer to the scripting world of Morrowind, i can't put myself in the position, and without a guide- or rather a history book how to use, modify or even look at it I feel there is no way for me to help you out.

You talk about using mwsql, and to active it with pipe line run a xFileWriteText function. I don't even know what this stuff is. And as far to my knowledge there is no real place to learn about it. I have knowledge in SQL, not morrowind scripting. So, sorry and good luck with everything!

Well MWSE comes with a rather awesome manual that explains a lot of this stuff, and Fliggerty and Yacoby are very generous when it comes to helping people understand a lot of it.
But, your reasons are understandable, and I'm sure we all can respect them.

I leart from the Manual and trial and error, however
http://www.tesnexus.com/downloads/file.php?id=13969 (In case you haven't got it)
http://www.tesnexus.com/downloads/file.php?id=20927

This has to be about the coolest idea ever.

Does anyone know if MySQL has a module/dll that can be linked into a program like I did SQLite with MWSQL? It could be possible to adapt MWSQL for this, keep most of the code, but use MySQL for the db engine.

As for learning MWSE, yeah, the best way is to look at the examples, read the function descriptions, and play around with it. Same way I learned C++...

Edit: Looking it up now.. I have a few ideas.
Edit2: I think it may be possible to use MySQL to connect directly to the server, or a local db. Not sure about either... I'll take a few days off trying to remove animation limits and see if I can fix this up for you guys.

A special gift for the thread http://www.msnheaven.com/content/emoticons/54/banana027.gif

Edit: Well, that was supposed to be a nanar... You get the idea..

That is fantastic peachy! :goodjob:

I've got my server set up with LAMP and everything is working fine now. So I'll be working on building the database and the general site structure.


I like the lines you are thinking along....

Yay! ^_^ Perhaps get rid of the "The" though, it's probably a bit unneeded imho.
So it would just be "Morrowind Census and Excise Office".

-wopz-