I know this can be a really touchy subject around here, and everyone has a different opinion on it. But I'd really like to have a conversation going on discussing what your idea of the "perfect" mod rating/feedback system is.

Now, I'm not just simply thinking about ratings in particular. But feedback altogether.

Some of the existing ones we see in this community that I can think of are:

• Open comments, anybody can add to
  
• Ratings on a scale of 1-10
  
• Endorse/Dislike
  
• Forum threads

What do you like and dislike about each of these? What can be done to improve them? And finally, what do you think is your ideal feedback system?

My main goal in creating a such a system would be to A. Provide detailed feedback to the mod author about bugs, conflicts, general experience, and praise; and B. Allow those looking for mods to get an idea of whether or not they ought to take the time to download and try the mod.

Obviously this is all subjective, but we do what we can....right?

Thouse 3 things would be perfect. A voting system doesnt really reflect what a mod is. Some people vote down because they didnt like a certai n part, others vote up cause they think the idea was good but the mod itself could be poorly executed etc...

I'm not a big fan of numerical rating systems. If you have few options it feels restrictive but if there are lots it seems many people still go for the maximums and minimums and leave the middle out of it. The latter systems, where you are rating out of 10 or 100, also tend to have much more personal variation in how things are weighed. For example I consider 40-60% to be a mediocre game, or a good game with significant shortcomings, but not one that is "bad." However I know other people who view these more like school grades so anything below ~60% is a failure that needs to be avoided.

Rating systems are also easily swayed when there are few votes and a score so a single vindictive rating of 1 or a few buddies maxing it out can skew things a lot.


If I were to implement a rating system it would need a key of sorts, an explanation of what each option/number entails in order to get everyone on the same page. It would also need to display how many people voted for each option in addition to what the average is. I'd also prefer a system that doesn't display the results until a certain threshold is met although this is admittedly a difficult thing for a mod site since some mods may have trouble getting ten votes while others may easily get a hundred.

However I'd be more likely to skip ratings entirely and leave it entirely to comments.

Here's a list of things that I like among the existing ones, and things none of them do that I think a good system should:


- "I use this mod" Allow me to mark the mods I'm currently using, and see who else is using it. This is the highest praise a mod can have from a person is use. Not "its good, I like it (but don't really use it)". You could even automate this with some mlox load order upload, or some such.
- ability to see which mods are being used the most by users.
- Bug report tool
- Review section - just for people's review of a mod, not support discussion
- Support section - area for help requests. I prefer to read this when I want to, not when I'm first considering a mod. I want to read review comments, then consider how much support people ask for and whether its worth it or not. When the two are intermixed, it clouds the situation (a lot).

I also vote "no" on any sort of numerical rating system. A quick look at PES (or Youtube) will show you just how flawed it is.

I think you have the right concept and balance for this sensitive subject around here. :thumbsup:

The only thing I really dislike is the endorse/dislike system they have on Nexus, because that could be difficult to comment a mod in a constructive way if you don't like it.

I agree with most people around here that Nexus isn't the best site when you're looking for MW-mods, but I do like the forum threads and the give kudos system they got there.
I know there is some forums out there which have a report thread available to any mod user, of course some of these forums do require that you must register before you can post your report. Only a few forums doesn't require registration so you can still post there.

I think the rating system for a mod could be almost anything and I'm quite sure that you can figure it out for your self imo.

I think the "I use this mod" option is the only rating system that will work as intended. After all, what better measure is there of how good a mod is?

>I think the "I use this mod" option is the only rating system that will work as intended. After all, what better measure is there of how good a mod is?

Yes. I think only a "thumbs up" option is needed. No "thumbs down".

Yeah, so you rate a mod based on your own experience and share your personal opinion/views to the author's right?

I don't mind the PES rating system. It may not be totally accurate to the quality of the mod but with enough votes (10 or more) you get a general picture (at least of how many people like it). Although the comments do sometimes get ridiculous, often turning into a psuedo forum topic than furnishing any useful info about the mod.

I'm not fond of the like/dislike, it's just doesn't give enough info to help decide whether its good or not (plus the site feel awkward to me).

I know this is coming out of left field, but I like the "rating" system from Little Big Planet. You can use generic "tags" to classify a level and you can "heart" (i.e. like) a level. So a level could be "clever, long, hard, unconventional, etc and be hearted 98 times". Not sure that is practical in this case or not but....

Again, none of this "rating" crap. In case you haven't noticed, we are an incredibly polite bunch of people (well, except me) and don't really wish to hurt anyone's feelings. Try as you may, any sort of rating system that goes beyond simply listing the ones you use are going to be fueled at least somewhat by that part of your brain that just wants to be nice. That's why things like blind taste tests exist - it's so that the tester's opinion won't be biased by the brand name.

It's like I always tell people when they read what I say on my mod list - the fact that I'm even mentioning it there at all is the highest form of praise I can possibly give.

EDIT: that came out a lot angrier than it sounded in my head. Please adjust the tone of the post as you read it accordingly >.>

I read the written opinions of players who have used a mod. I ignore anything else.

I certainly don't want a heat discussion about a rating system. No I don't want that, but I suppose you took my comment to serious. My apology if you did.

Haha, nah, I was just trying to make a point.

And I apparently always sound angry when I'm typing.

So what about a comment system that splits your comment into one of several categories based upon your activity?

If you've downloaded the mod, your comments will be displayed among those others who have downloaded it. If you have not downloaded it, it will be displayed separately again. Perhaps each viewer can even choose to turn off display for comments from those who have not downloaded it in their profile?

I also really like the idea of separating some feedback out into different sections, ie bug reports, support, reviews, etc.

That would be cool, it would help tidy up the comments and make moderating the comments easier.

I always wondered if a 1 - 10 system would work... but you alter each users votes so they are in a standard normal distribution. So that if someone never gave any mod less than 8, if they vote on a mod with a rating of 9, it gets translated into something like 5 (Assuming they have equal votes to 8 and 10)
Of course, that would probably annoy mod makers because it would lead to the majority of mods getting between 4 and 6 out of 10.

While probably a more accurate measure of what other people think of a mod, it does have disadvantages. More of a 'I wonder how well it would work', rather than a suggestion. Overall I don't feel like there is a much of a difference between a 1 - 10 system and a binary system due to everyone having different ideas on what constitutes 10.

(From experience, for most people, it is almost always 10, unless there is something wrong with the mod, when it is some value less than 10). This isn't helped on places like PES where voting isn't anonymous.


Another model would be something like Stack Overflow. Each user has a certain amount of 'karma'. Other users can vote up and down comments and mods. Down voting would cost karma. But upvoting would be free, and net person who wrote the thing that was upvoted, karma.
This would lead to a system that favoured voting up mods (Mod makers like upvotes) without leading to a loss of anonymity or having to write something (TESNexus).

Lets say, an upvote on a mod nets the mod maker 30 karma. A downvote on a mod costs downvoter 30. An upvote on a comment nets the commenter 10 karma, but a downvote costs the downvoter 2 karma.
That way, it hopefully encourages feedback when downvoting mods because if other people agree, you can recoup the cost.

You would end up with people having made 1000s of up votes, but maybe only a 100 or so downvotes.

Interestingly, on Stack Overflow, downvoting costs 1 reputation, where as an upvote nets the person 10. Because reputation is something you try and accumulate, most people don't downvote much, if at all.

This is in contrast to somewhere like Reddit, where down voting has no cost so the number of downvotes is often about 1/3 - 1/2 the number of upvotes.

One key advantage of this system is that you can 'trust' high reputation users so you can have semi moderators (Maybe with the ability to flag bad comments, and if the comment reaches 5 flags, it is automatically deleted).




Another thing would be to have a bug tracking system (Which I am really confused as to why mod hosting sites don't have). So rather than downvoting for a bug you would encourage the user to open a bug report. If you displayed the number of open/fixed bugs, then it would show how well the mod is put together without really explicitly being negative feedback.

Yes you did make a point and you don't sounds angry when I read your replies, but of course that's just me. :wink_smile:

I couldn't find a better word than "rate" when I comment it with "...based on your own experience..." expression.

I would assume that it's a given to be required to download the mod before you, you know, review it.

The only sort of comments I could even imagine should be in any way allowed at that point would be about the reason they didn't download it (i.e. they disagree with the concept of the mod as a whole)



And people who are posting about a mod that they haven't even downloaded



Fair enough. The way you type makes me read everything you say in the voice of Toki Wartooth. It's quite amusing



Yeah, I think my main point is that the idea of a "rating" system should be abandoned in favor of a comments section, whereas actual "rating" will be measured based on the mod's actual popularity with users.

Which would mean you need to have an account to download, which I am not really in favour of. You want to make the initial effort as small as possible to help it grow.

Of course it is. But that doesn't stop people. That was the most common reason for removing comments and votes when I moderated at PES. We'd get comments like "Dumb idea, won't even bother downloading." And it would have a rating of 1. Obviously they didn't download it, so shouldn't have given feedback.

It would be simple enough to flag a user as having downloaded a particular mod, and then separating their comments based upon that.



And that is another point that makes it difficult. I also agree that guests should be able to download anything. But I have always made it a point to require membership in order to really contribute. Which is why I would lean towards requiring membership and a flagged download before being allowed to leave a comment/review, while allowing separated comments that bear no weight upon rating/kudos/whatever-the-heck-is-used from others.

Not quite. There *is* another way.

I am reminded of a little hack utility that someone made for another game some time back. When the program started, it asked you if you had read the readme. If you replied "no", the program kicked you out. [censored] hilarious.

Something similar could work here. You've already stated, Fliggerty, that you want to separate the comments into their own sections. And one would assume that "people who haven't downloaded the mod" would be in its own section, since they are going to be comments based entirely around the concept of the mod rather than how it works. You can leave comments like that without an account, and thus can download without an account.

But, you'd need an account to leave other types of comments.

Good idea Fligg. :nod:


I think the golden era of guest account policy is long gone by now, especially thanks to the spammers these days.

The staff at GHF/MMH and I already have our plates full without having to constantly remove spam comments. I would rather one legitimate comment not be posted because someone doesn't want to register than have to remove 1000 spam posts selling me viagra.

So what would you think of a system that gave each registered user the option to click a button on each mod that says either "I use this mod" or "I no longer use this mod?" Do you think that would provide a decent reflection of who actually utilizes a mod? Would you come back and click such a thing for a mod you use?