wow, i'm gonna have to get that custom face and hair i've been working on done, so i can show it off on the website

oh and whole thing with downloading a persons character to fight against it sounds fun, but i'd be too tempted to upload a 60ft lvl 255 argonian equipped ready with his trusty daedric claymore, enchanted with damage health 200pts for 200 seconds in 100ft and a spell that summons 10 golden saints

maybe something like this, which joins the community which plays the game together and generally makes the whole thing a lot funner, will actually get more people playing the game again, and it's the next best thing to multiplayer (which would screw up the game, imagine you both reach the cavern of the incarnate, it would be a race to see who get's moon and star, i'd probably just throw a daedric dart in the other person's back and laugh as i pick up the ring)

Zarnatheron's suggestion for handling inventory, stats and spells is a lot better then mine. I second this.

This is the most interesting idea I have heard in a long time. If anyone else would have suggested it Fliggerty I would have just disregarded it. But if you actually get this working, I would be very tempted to upload a character of my own (I would have to manually upload it, no internet :brokencomputer: ).

Great idea, it's no wonder you are receiving such a good reception! So I suppose this post is just a support and good luck post

-Melchior

Connection security is not an issue, as you are not submitting any personal data.
Database verification for the server would be done at fliggs end. There is no point doing it anywhere else, as they can be avoided.
There is no data coming downstream
Even if it was, it is one terrible program desgin that allows evaluation of code from a remote source.

I sure hope I don't have to cut this into two posts as well. :hehe:


An interesting idea. Seems like something that will need to be implemented once I get the basic idea working though. Right now my model is based upon overwriting data as it is updated; but that goal is something I will keep in mind.


Fantastic! As it is, I have a blank slate; so I will design it around what you build. I suppose for dimensions something like 200x700 px would work well enough. I don't yet have any ideas for the actual look of the place, other than it needs to say "Morrowind" somehow. I would like to keep it different than the Morrowtheme I have as default at GHF though. You are the artist, so I trust your judgment. This is to be a community-endowed project. Perhaps some of the in-game signs (and all of the beautiful retexes out there for them) could give an idea of how a sign for a census and excise office should look. The only requirements I give is the text; I want it to just say "Census and Excise Office." Depending on how it looks complete, I don't care if you feel that using & instead of "and" is better, just so long as the title comes across somehow.

Thank you! :hehe:


Fantastic idea. Something that would be implemented once this is up and running though.


Indeed, something that ought to be done. I'm not entirely sure the best way to go about doing something like that, but I have given it thought. I was considering adding things from some of my mods such as number of revenants raised, or number of duels fought. Modding is the name of the game here, so will have to figure out how to make that work. I need to learn a bit more about how MySQL works first so that I can allow for infinite growth like this.


I think this is obviously a major concern for me since GHF was one of those hit. I have become quite paranoid about security; that is one of the main reasons I set up a test server at my house, so that I wouldn't be leaving any holes during development.

I am paranoid about SQL injections. Google has a lot to offer though, I spent considerable time yesterday reading a lot about PHP + MySQL security. I will be taking every precaution to keep this as safe and secure as possible. And I am hoping that as far as individuals users download the necessary software and register with the site that my personal reputation around here will contribute to their trust of any endeavor of mine.


The user will have the option to edit many things, especially mod added aspects of a character. I believe that the setup process in the mod itself will allow the player to input such things; alternatively they can be edited on the site itself.


Thank you, such a compliment means a lot. I've always prided myself on picking up such things quickly. C++ was the exception though...I just can't wrap my head around it for some reason. Although PHP makes perfect sense to me. :shrug:

I believe that the Users table is going to be a bit different. I've been going through a tutorial about implementing an access control list, and the structure suggested for that is a bit different from what I suggested. I will be modifying things a bit to account for that.


Wonderful suggestion. I have noted it.


Does doing such a thing help improve speed and performance? Or is there another reason that doing this is suggested?

So you are suggesting that I make a separate table for data that is collected from the website itself? Should I then include things like the CreationDate in this second table?

Post #2


Another good idea, seems to simplify and complicate things at the same time. My type of change! :goodjob:

I also jotted down your suggestion for the vampire change and the faction table. Upon thinking about it, I do think that it may be best to keep faction information in a second table. As such, it is probably best to put the vampire info there, as the clans are technically factions.


The problem with this is the lack of scripting functions to determine what slot something equipped belongs in. I can find what is equipped, but that's about it. I have worked out a complicated method for determining armor type, but it is overly complicated and intensive to be run as often as this will need to do. So I think we will have to settle for simply listing what is equipped, divided by armor and clothing. I would hope that MWSE will fix this in the future.

As for the rest of the inventory, I actually think it will be best to maybe have it all listed in a single text file.


So you say that it would be a single row for each NPC? That would get pretty big too. What if rather, this table was changed to represent the current cell, with 20 columns for hostile NPCs and 20 for friendly? That would allow for the data to easily be overwritten with each update, and probably avoid more bloat.


Thanks for the suggestion. Probably a good thing to do indeed.


Yes, it is. I'm using Ubuntu Server 8.10 Intrepid Ibex. Apache 2, and the latest MySQL and PHP 5.


That would be fun indeed. Again though, this is one of those things that ought to be added once the basic function is implemented. It would actually be quite easy for most things too; all that would be needed is to check journal entries.

One thing I did allow for in the initial database idea is a custom message from the player. I will implement a way for the player to initiate a text box that they can type a message in. When they do, it will be sent to the database immediately and displayed on the character screen. So they could something like "Here I go, to face Dagoth Ur!"


I do like that idea. It could easily be divided into MW, Trib, and BM sections, and the highlights are displayed when it is done. Something to think on.

And #3. :hehe:


Beyond my current capabilities, but that can always change. I can tell which direction the player is facing at any given time, so long as they are in an exterior cell. That might be one more bit of data to add.


Oh there is, I called it "Caption."


Wait, you can do that? I didn't know you could put a table in a table. I'll go learn how that works.


Unfortunately I can't easily tell if a skill is damaged or fortified (unless the player is GCD, then it's a bit easier.) Only with attributed. Another thing I hope to remedy with the next version of MWSE.


And spells are another problem. Without running a script that checks for each and every possible spell in the game (which wouldn't be mod compatible) I can't find what spells are on the player. I can determine which spell effects (for some of them, the function doesn't work for all,) but not the name nor magnitude of the spell. I'm still unsure how best to handle this. I may end up limiting spells to only racial ones, or just allow the player to input the spells themselves.


Thanks for all the kind words everyone! :goodjob:

I haven't looked at MWSQL, but if it uses normal sql or something similar, then the principle is that, you have a mastertable, which you could call it character with a primary key, which should be system generated to ensure, that it's unique. This table could also standard information like character name, race, level, birthsign and maybe also other information (the vampire example is an efficient way of storing the information, but remember, that you have to both encode it and decode it).
Then you can have the childtables, which are tables as well. They have a row with the value of the primary key for the character it belongs to, so you easily can fetch the data to the character.

With as many rows as you mentioned in your first post, it will cost in performence, while extra rows aren't that expensive as long as the subtables are indexed correctly (using the row with the value of the primary key). It should also be easier to get an overview, I believe.

I hope that made sense.

G/l With this fliggerty Its gonna be hard..

As a natural-born showoff, I love this idea

Of course it is...that's why I love the idea! :hehe:

How about an Object Definitions Export for all equipped items, complete with media file extraction and bzip compression for storage in "blob" records associated with the character?

Haha! 3 posts!


It's not primarely about having only one source of updates. It's mostly about the PageViews column that has to be updated whenever a character is viewed, which means every reading query on the character table, implies a writting query on the same table. So your amount of writes on the character table will always be greater or equal to your amount of reads.

If you go with the replication solution with replay logs, you'll have to be careful of your write/read ratio on the same given table. The lower, the better. Replication performs best when the amount of reads is greater then the amount of writes. Otherwise queries tend to pile up in the process list, depending on how much queries you get and how fast your server can treat them.

It might not become an issue. You'll have a better idea of how it performs when you get to do some tests with live testers playing the game while other live users browse the website.


Yeah, but if you go with Zarnatheron's suggestion of splitting stats in a separate table, which I think is better, it won't be worth the hassle.
You can always apply it somewhere else. It's a handy way to stuff lots of true/false flags in a single variable or table column.


Re-reading it your original post, I realize you meant columns. So my thing isn't a good idea after all. You'd better stick with your original idea.


Nice.



No tables within tables, but tables relating to an ID in another table. Exactly like you already did with your Users table and your Character tables.

I got ahold of a little mysql distro, and it seems to have the required files to make.............

MyMWSQL

Just need to get the hang of the new API, change the pipe name. Generally it'll be the same, but this should save everyone quite a bit of time from developing new programs. I also got cURL, and am looking into it. As soon as Code::Blocks works right [kicks it]

It will, in the end, interface with databases similar to MWSQL. The problem is I can either have a MySQL embedded server or make it solely client (all that is needed here is client, so this could be a special case deviation).
SQLite, it seems, is far better for the goal of MWSQL. However, the code can be (almost completely) reused for this.

@ Fligg: if you have any special feature requests, and you wanna use a variation of mwsql, I'll see what I can do.


MWSQL, at least as far as queries goes, is standard SQL. It only wraps SQLite and allows communication, it does no internal query processing.

A minor update:
- I looked at the MySQL more in-depth.
- All but the result handling code of MWSQL has been ported to MySQL, renamed MyMWSQL. I actually got the row count functional as well.
- It appears cURL won't be needed with the MySQL API, it can connect directly to the server (or from settings in an ini, if that's easier).

So if you, Mr Fligg, do choose to use this for your client-side app, it's almost done. If not, I've learned a little more.

Edit: I'm trying to get the code for both versions up, but am having a [censored] of a time with Subversion.

Ah, ok. I understand now.

I think that I will be breaking the character table up a little bit. The factions will go in a separate table, and I believe the "current data," such as current cell etc will be another one.

Sounds great peachy!


I do intend to use MWSQL, the whole plan is based around it.

There are two things that I would like to get which it doesn't currently do (primarily because that is not within the scope of MWSQL.) I want to be able to read the mod list from Morrowind.ini, and also send the journal.htm. Is there any way you would be able/willing to perhaps come up with a way to facilitate that? :icecream:



Okay, so we can have this either query a local database, or a remote database? So we could then give the user the option to just directly write this to our online database without wasting the time and effort of writing it locally first, as Yacoby originally suggested. Or they can change an ini setting and store it locally to be able to upload it later? That would be the best of both worlds, without any compromise at all.

Although, I do have one concern about having it written directly to the online db. It would be fairly simple for someone to make a small program that writes to your pipe (or even use MWSE I suppose) and call all sorts of queries, correct? So this would essentially give someone the option to access everything in the database if they query the right things. Security is an aspect of this that I haven't entirely delved into, though I will before this gets much further along. I had assumed that if we had something running server-side that would accept the data from the client and then write it in the db that there would be the opportunity to check for SQL injection and other such things. But if we are writing it directly from MW, that option isn't available. Does anyone have a suggestion for this perhaps?


P.S. I just want to take a second and complain about "Authorization Mismatch." Anyone else been seeing that an awful lot lately? I can't type more than a few sentences without the need to reload the page before I can post. I sure wish they would change that back to how it was.

Yes, that and a time out wait before I can send a second PM.

Thought I could comment better on that than MWSQL.

The idea really appeals to me. Nope I can't do much to help, but I'm interested in to somehow take part in it...

Reading the INI should be no problem. I'm pretty sure there's code for that in the windows sdk, so I'll just use those functions.

Uploading the journal is trickier. You'd either have to pass the file or read and pass chunks, or have a huge field in the database. I don't know if it's a good idea to keep journals on your site, but it's possible I think.



Unfortunately, I don't know if the MySQL part can access a local database. Which makes it incompatible with MWSQL (kinda, or would have to run on another pipe next to it). Unless both are combined. I'm not exactly sure, but I'm starting to think having an external app you run and uploads the data could be far simpler, at least for users.
How it works now, you have MWSQL, using SQLite and a local database. Or MyMWSQL, using MySQL (client only) and a remote server and database. The former is perfect for storage by mods, the latter is useless, as it's forced off-site. You would either have to have a MySQL server included or on your system, or something along those lines. Or run both. Horribly complicated.
Writing the data out to MWSQL/SQLite, then having an app read that data (or the savegame directly, as it contains everything) and send it up to the server using MySQL would keep one app running in the game and probably simplify things quite a bit.


That's a possible problem. My testing program, actually, can write to the pipe (lets you test things without opening the game). That would be all that needs done.
You would have to set up a filter at the server to help with that, or (probably a better solution), upload the data using post (and cURL) to a webpage.

That's the best solution. Just popped into my head. Essentially, the data is written to MWSQL. No MyMWSQL exists, so everything is fully compatible. Whenever you want to upload your data, you run another program, pick a savegame (or database). It read that in, translates it into various fields for a POST, and opens a webpage. The page (written in PHP with filters, only accepts data, no requests and returns, which I could set up for you) adds/updates the character in the database and takes the person to that char's page.

It's essentially perfectly secure (except for huge numbers of chars being added) since the data being sent is strictly limited to what the page reads in from the $_POST array in PHP.

Edit: That's confusing. Here's a list/chart:
- Player saves game
- Player closes game
- Player runs upload program
- Upload program reads game, taking certain records and adds them to a POST query
- Upload program sends the request and POST data to http://fliggerty.com/addchar.php
- addchar.php takes the data, reads certain fields from the POST data, and updates the database

All login information for the database, even the name of the database and table names, is kept on the server and protected by PHP and Apache. Similar to every CMS that exists, and should be just as secure.
It requires one program run by the player, and three pages on the server (add/update character, view characters list, view this character). One database, one or two tables (stats and inventory).

Maybe just have all "submit" operations handled by Stored Procedures? That way you could abstract the interface to minimize client-side changes as the database evolves, while validating input and performing add/delete/update operations in a controlled environment.

It's not such a bad thing to allow arbitrary SELECT queries though, maybe someone will want to write custom client apps for as yet unforeseen purposes?

Heh, that was what I suggested in the first thread (Well nearly)

Although you don't need to open a webpage, you can just send the request to the server. IDK if that is what you meant by 'opens a webpage'

You may want fields in the local database that contain a boolean value if it has changed since it was last updated to the remove server. Cut down on the bandwidth costs, although it may add too much complexity to the MW Code side, which doesn't handle added complexity well (i.e. You can't generalize it into a function). Deppends on the design. :shrug:



@Fligg
If you are going to do a big app in PHP, look up the MVC (Model–view–controller) pattern. There are a load of PHP frameworks, and then there is always stuff like Rails.

@ Shanjaq:
If things were updated through a PHP page, any updates needed in the future could be easily modified for that, as could sending data.
That sounds like a much more secure interface, and it would be relatively simple. Old versions could be told to update or be kept compatible, while new ones would supply all the right data. The PHP would take care of updating or adding, splitting, and building the query.


@ Yacoby:
Yeah, well...
I meant send the request, not exactly open the page. Unless you wanna do a profile type deal where it opens the page with your new info displayed.

I'm not sure about how to check for updated-since part, though.
Say you have a program that reads the save file and sends the data. Well, you would have to check each field against a stored copy, doubling the space. Or against a checksum of the previous field. Or something. I think checking the save or the database generated for a new checksum would work better, although you would update all the fields.

Even Morrowind uses those functions to read its own Morrowind.ini - GetPrivateProfileSection, GetPrivateProfileString.

Indeed it does. They seem to be in the docs, too. I'll just use those in whatever app ends up doing this job.
It's also nice how many things MW calls from DLLs. That's the only reason using the My Games folder could be possible, since the import table can be changed around.

In its most basic concept (ie take up loads of space, is crap), when the program that syncs the data would create a copy of the database when it uploaded the data, next time it uploaded the data, it would do a comparison check with the data it had previously uploaded (stored in the copy of the database), and only upload the changes. It would then copy the current database (now the same as the one on the remote server), overwriting the previous copy.
Of course, there are more elegant solutions...

Trade off with bandwidth usage / upload time and storage space on the users harddrive.