Upcoming Beta - Feedback interaction?

Post » Fri May 29, 2009 10:58 am

So everyone's getting excited about the upcoming Beta, and everyone has no doubt ticked the box and are waiting anxiously to find out what's going to happen when the time comes. People are looking at what to expect in the game, people are asking lots of questions about what's going to be in it, but I've not seen any real questions asking the big one (Unless I've missed it):

What plans are in place for taking feedback and construtive criticism and applying that to the production?

I have Beta'd several MMO's and have always been a fairly involved tester. I believe it giving large quantities of quality feedback, suggestions and ideas. Yet numerous times the developers make no allowance for this sort of input, this sort of feedback, and after a while of being bombarded by the testers on the Forum, simply stop listening.

Don't get me wrong, I understand that there are some very mouthy people with very poorly formed opinions and ideas, that's to be expected. Their comments can sometimes swamp the sensible, thought-out ideas and suggestions put forward by others.
Yet with a game so monumentally important as this, I would hope that a system is being developed where there is enough scope to give back co-ordinated feedback and input. Often, sticking it in a Forum thread, whilst great for community discussion, ends up with poorly laid down ideas or it descends into chaos with aruging and bickering.

I'd be interested to know what ideas are in place for taking feedback from testers when the time comes, and compare it to how other developers have gone about it.
It's always a massively under-estimated area. Here's hoping we avoid mistakes other companies have made in the past.
User avatar
Princess Johnson
 
Posts: 3435
Joined: Wed Feb 07, 2007 5:44 pm

Post » Fri May 29, 2009 8:38 pm

I'd like to see an ingame bug reporter. A bit like steams support reporter, where you can choose directions of the bug etc, if you dont know what im talking about, go to steam support and just test :P

Not exactly like that >,< but more like: Bug report, what kind of problem is it - **open list, selects issue** - connection issues - write your headline and problem.

you know.. like that but a bit more stuff >,< just a small example, this way all their bugs will be automatically categorized, I just hope they are way more faster than Valve at answering and only bug reports are allowed to be reported, any small questions will be disregarded and a spam warning will be sent to the pm:er, more warnings will lead to timebased banning.

Also when sending you can select how big of a problem it is - Low - Medium - High, so all high, if they lie about the gravity of the problem a warning is sent.
Like: connection issues, crashes at start up all the time - High.
graphical issue, character clips at the chair when sitting - Low.

Also, there should be a place where people can see all these bug reports, so if they encounter a bug they can first search for it there to see if it has already been reported, then they could be able to press a "thumbs up" that means that they also have the same problem (no names are visible for anyone except for devs/bug handlers). The thumbs up also adds your name to the bug report. Then there could just be a normal forum part if you have any small questions, or a mail address, at least I think the bugs are more important. Image
User avatar
Mr. Ray
 
Posts: 3459
Joined: Sun Jul 29, 2007 8:08 am

Post » Fri May 29, 2009 12:27 pm

I agree having a streamlined ingame report would be awesome. Sometimes using the forums are a bit of a pain. I have also been apart of quite a few beta tests. The current one which is still under NDA doesn't have the greatest ingame report but its better then other games I've beta tested on. The original World of Warcraft beta was also pretty good, but the forums on the other hand was so horrible filled with garbage I rarely used them. I think for the most part about the forums is the fact that a lot of people that get into the beta shouldn't. Either that or they are to lazy to use the correct forum or report bugs. A beta is not a demo and it ticks me off to high hell when companies use the beta for a demo.
User avatar
clelia vega
 
Posts: 3433
Joined: Wed Mar 21, 2007 6:04 pm

Post » Fri May 29, 2009 8:47 am

As a professional software QA test consultant, I feel feedback is the key to refining a product.

Beta's are excellent in flushing out large numbers of bugs, and automatic crash log reporting would be an excellent time (and cost) saving feature. To have large numbers of people trying to create minidump files would be next to impossible. At the same time, the logistics of this are staggering to anyone who has never worked in this professionally (at least when I try to describe what I do to others).

I have to say that the suggestions in this article are spot on: http://www.developer-resource.com/beta-testing-tips.htm

That being said, a system where individuals get to rate the importance of their own bugs, almost always ends up with people believing the issue is critical. I believe a better system would be a rating system of peoples reports in house. Might alienate some tho, so the politics of this would be touchy. In any feedback system I would say that the best thing you could do is design the system such that it automates as much of the report as possible (build versions, game settings, etc) for the reports. This simplifys the job for both the developers, and the volunteers/fans.
User avatar
JESSE
 
Posts: 3404
Joined: Mon Jul 16, 2007 4:55 am

Post » Fri May 29, 2009 7:02 am

Reference the comment about people rating their bugs by level of importance, the easiest way to resolve the problem of everyone thinking theirs is URGENT is to incorporate a set of check-box questions, which if the right number / selection of them are ticked, rates the bug as per instructed.

So:

1: Does this bug render the game unplayable - Y/N
2: Does this bug cause system instability - Y/N

And so on, and so on.

Automatic crash logging is a must, never mind the fact that many people wont bother with it if they have to do it themselves.

Reference the comment about the wrong people getting into Beta stages..

Well, it's a tricky one that. You need numbers to test things through, and so the muppets that constantly scream about silly things are at least serving the purpose of playing the game. Guarantee if there's a bug that causes serious problems, they'd be shouting just as loudly. However I do agree that too many people get into Beta's so they can demo the game, smash around whilst there's loads of glitches and 'cheats' and just have a merry old time.
Although these people are often instrumental in getting a bug fixed, without realising it. If they start taking advantage of a glitch which gives 'xxxxx' bonus over and over again, they'll do it so much, that it will become common knowledge and thus get reported up eventually.

With regards to actual suggestions for game features and feedback on that note, I strongly feel that a feedback form should be used, with plenty of drop-down menu boxes and check boxes so you can refine what it is you're suggesting (To make it easier for the dev's to sift through) and then plenty of space for you to break down in detail, what you are suggesting, how it would work, what it would achieve.

If you didn't fill out all the boxes, it doesn't get submitted. The devs then have at least a half-thought through idea that they can look at, say; "I like that one.." then put it on a strongly controlled board on the Forums that the rest of the community can look at and vote on it, or offer suggestions.

The catch is getting an idea noticed in the first place, which this system would resolve, and keeping it controlled. As you have the initial input, it's not hard to remember where you started, and as good suggestions are added to it, it can be ammended.

That's the biggest bug-bear I have. Companies developing games that lack features which the testers screamed for at the start, and which were entirely feasible. If someone is suggesting something, it's because they want to play it, so if one person wants to have that in game, rest assured another few thousand will (Unless it's a mental idea!) and so you're looking at selling more games when people find out you can do 'xx'
User avatar
Taylor Bakos
 
Posts: 3408
Joined: Mon Jan 15, 2007 12:05 am

Post » Fri May 29, 2009 12:56 pm

I prefer using the Bugzilla to report bugs. It has simple and intuitive interface, but at the same time it allows to specify the bug by some categories and control it's solving by built-in feedback. Excuse my bad English... I am a Russian bear, it's hard to type with claws.
Image
User avatar
Jeneene Hunte
 
Posts: 3478
Joined: Mon Sep 11, 2006 3:18 pm

Post » Fri May 29, 2009 7:14 am



Very much this, especially about the points related to bug submission and the lack of submission for incomplete stuff.

Going through betas for some other stuff, people sometimes don't fill out incredibly important parts of those submission forms, or worse, they fill a lot of them out as such, and then it's just cluttering up space instead of helping the devs.
User avatar
Claudz
 
Posts: 3484
Joined: Thu Sep 07, 2006 5:33 am


Return to Othor Games