Do these games ever get tested?

Post » Wed Feb 03, 2010 9:59 pm

With every player that I have talked to personally on different versions of the PS3 having issues, I have to come to the conclusion that no one even bothered to test this game on a PS3 before shipping it off to the stores.

It isn't possible for anyone to have played this game for more than a few minutes without running into some of the more minor issues. And after a few hours it would be quite clear on any machine that this game was not even close to ready for shipping.

I was pissed when these same types of issues were found in Oblivion. But when Fallout 3 came out and there were fewer issues (but still too many) I thought well, maybe they are getting better. But with New Vegas having so many bugs that most of the quests get locked up at various points, it is clear that Bethesda is in a "Give me the money and shut up" frame of mind about their products.

I can't think of another game to hit the market with so many bugs in it. I think Bethesda should get some sort of award for the "buggiest software on the market".

I know they say they are working on patches, but that is not the point! The software should have never been released with these massive issues. These games are the only ones I have encountered that can actually lockup a PS3 to the point that the power must be shut off.

Come on people! If you don't have the budget to do some basic testing, I would bet you could find some beta testers willing to play the game from beginning to end for free on real world PS3 systems and do full reporting on how each bug was encountered. Heck, I wouldn't gripe if this was a beta but it isn't it is a production release that is of lower quality than most beta software I have seen.

Ok, done ranting now. I tell you though, if we could return our software under the heading of "doesn't work as advertised", junk like this wouldn't hit the market because the cost of the returns would out weigh any potential profits.
User avatar
Sabrina garzotto
 
Posts: 3384
Joined: Fri Dec 29, 2006 4:58 pm

Post » Thu Feb 04, 2010 4:41 am

Yes, these games are tested, although, I did hear that this game only had like 3 testers. Anyway, this type of game is so voluminous that there are literally thousands upon thousands of possibilities with how different events can interact with one another and there just wasn't enough time to test every which way. Hence why the game was so buggy. IMO games like this (as opposed to something more linear, like MW2) need dozens of testers to test the game every which way imaginable for at least a couple months, but they also have to meet a deadline. Now on the other hand, a game like MGS4, that game didn't even have a patch (well, it did, but it was a trophy patch, not a bug patch) because it was so vigorously tested. The game was flawless.

PS3 users have it the worst, bugwise, then 360 users, then PC.
User avatar
Emma Louise Adams
 
Posts: 3527
Joined: Wed Jun 28, 2006 4:15 pm

Post » Wed Feb 03, 2010 9:34 pm

Yes, these games are tested, although, I did hear that this game only had like 3 testers.

I seriously doubt the game only had three testers, although it's possible the person was referring to the fact that the game must have been tested by three groups: Obsidian/Bethesda, Microsoft, and Sony.

From my understand for a game, DLC, or patch to be released on the XBox or PS it has to go through MS's/Sony's own QA process and be certified.
User avatar
Rex Help
 
Posts: 3380
Joined: Mon Jun 18, 2007 6:52 pm

Post » Wed Feb 03, 2010 8:54 pm

This game is so very big, it's impossible to see every glitch, every error, etc...
I think they did a realy nice job for a game this big!
User avatar
Tanya Parra
 
Posts: 3435
Joined: Fri Jul 28, 2006 5:15 am

Post » Wed Feb 03, 2010 10:03 pm

I agree that it is impossible to test everything in a game this big. But given the fact that I know 6 people that have purchased the game (including me) and all 6 of us have hit very serious bugs within a few hours of play, it is clear that no one actually tested this game in a reasonable way.

It would be no big deal for a hand full of people (say 10 at a minimum) to play the game on a basic PS3 for 20-30 hours of game time.

It is statistically impossible for me to know personally 6 people that all encountered bugs if they were limited to the unusual twist of fate in the game. And it is also unlikely that so many people would be seeing the same bugs in the same place if the bugs detection required a specific set of events to happen in a specific order.

As you said, there are so many possible variations that there is virtually no way that all 6 of us caused the exact same circumstance thus these bugs must be simple to detect. Perhaps not the exact same bug but the same cause/effect.

The testing that Sony and MS do relate not to how the game performs but are they secure and do they follow the programming rules they set in place. It is not MS or Sony's job to say that a game is or is not of good enough quality to be sold, only does it meet the minimum specifications to function on their consoles.

The cause of these lockups is most likely routines that get caught in infinity loops without any CPU interrupts. Otherwise the consoles would be able to regain control and kill the game but the lock-ups are such that the OS can't even regain control.
User avatar
Julie Serebrekoff
 
Posts: 3359
Joined: Sun Dec 24, 2006 4:41 am

Post » Thu Feb 04, 2010 4:37 am

Oh one more thing... These bugs are not limited to the PS3 but impact all platforms.

And it would be so simple to find a bunch of people willing to BETA TEST the games likely for FREE before the game went to the stores. Even if the BETA testing was to happen while production was starting up, then that first patch we got on day 1 would have covered so many more of the major issues.

I wouldn't be so pissed off if I was beta testing this game. It would be expected. Heck, I would beta test for free.
User avatar
Rude Gurl
 
Posts: 3425
Joined: Wed Aug 08, 2007 9:17 am

Post » Thu Feb 04, 2010 11:56 am

According to the credits there seems to be three internal testers at Obsidian and ~80 at Bethesda (since its the publisher duty to test the builds/game).

I am also curious how the test process of such a game is, it felt like they just tested the areas which were added with every build, flagged them as okay and moved on. It seems like no one played the game from start to finish or did multiple playthroughs.
User avatar
Tom Flanagan
 
Posts: 3522
Joined: Sat Jul 21, 2007 1:51 am

Post » Thu Feb 04, 2010 7:00 am

I'm really concerned that this game will hit microsoft/sony's (heavily)rumored patch limit very quickly, leaving us with the game in a sorry state. The fact that Bethesda refused to patch Fallout 3 and Oblivion GOTY editions leaves a very dim glimmer of hope for even a quasi-polished New Vegas.

The pessimism towards Bethesda products is truly not undeserved. They have a proven track record of lackluster support for their console releases.

I'd like to add that I'm back here posting because the game just hard locked on me for the third time in an hour, and I'm taking a break before I sink low enough to becoming truly angry at a video game...
User avatar
Sara Lee
 
Posts: 3448
Joined: Mon Sep 25, 2006 1:40 pm

Post » Thu Feb 04, 2010 1:46 am

I'm really concerned that this game will hit microsoft/sony's (heavily)rumored patch limit very quickly, leaving us with the game in a sorry state. The fact that Bethesda refused to patch Fallout 3 and Oblivion GOTY editions leaves a very dim glimmer of hope for even a quasi-polished New Vegas.

The pessimism towards Bethesda products is truly not undeserved. They have a proven track record of lackluster support for their console releases.

I'd like to add that I'm back here posting because the game just hard locked on me for the third time in an hour, and I'm taking a break before I sink low enough to becoming truly angry at a video game...


I know just what you are saying. I am worried about getting patches too. But other games I have are patched all the time. Normally for upgrades rather than bug fixes but what type of patch shouldn't matter and I can't imagine MS or Sony saying "We don't care if it is broken, you have reached your patch limit" because that would shift the problem to them making their consoles look really bad and makes the problem software company look like the good guy by letting them say "We have the fix but MS and Sony won't let us give it to you." and that just wouldn't happen.

I am sure they do "Unit Testing" which is the act of testing each unit of code. I am sure they do a great job of that. Many companies live by the unit testing model and are total morons if they only do unit testing. Their thought is that if each unit (module or discrete block) of code and it works, then they will all keep working when they are put together. Unit testing is fantastic for the most part but it is no replacement for field testing once all the units are put together. Has anyone found a single person that has played the game for more than 20 hours of game time and not encountered any major issues? Serious pausing, artifacts, floating creatures, getting stuck inside of objects or (the worst) lockups? I talked to three more people yesterday at the game store and they all hit problems. More proof that no testing was done on real systems using regular game play. Testing done on dedicated machines "perfect" machines, simulators and unit testing don't count. People have to actually play the frigging game to test it. It is called USER TESTING and is the only true test of software.

I have 13 quests that can not be completed due to very repeatable lockup points. If I even venture onto the strip at this point the game hard locks. I managed to get to the strip using the monorail and that did let me move around for about 10 minutes then it locked. Now I can't get back to the strip through the North gate nor the monorail because the strip is closed to me due to consistent locking.
User avatar
jessica Villacis
 
Posts: 3385
Joined: Tue Jan 23, 2007 2:03 pm

Post » Wed Feb 03, 2010 11:54 pm


I wouldn't be so pissed off if I was beta testing this game. It would be expected. Heck, I would beta test for free.

I guess you haven't realized yet: you ARE beta testing this game.
It just wasn't free; you had to pay $60. And don't offer to do it for free on the game's web site: that's why people don't get paid for beta testing anymore. Now they hand it off to the users like it's some sort of prize.

To tell you the truth after seeing their history with this sort of thing, I'm not very optimistic about the future of this title either. We've already got one patch that doesn't seem to have fixed the game breaking bugs, just the more amusing ones, exploits, and stuff like that. Just like what they did with Fallout 3.

I guess we should be happy we're not getting the dlc right away. It's probably going to render the game completely unplayable.
User avatar
Darian Ennels
 
Posts: 3406
Joined: Mon Aug 20, 2007 2:00 pm

Post » Wed Feb 03, 2010 8:40 pm

Once I realised that Berthesda had asked Obsidian to do "Fall Out New Vegas" I thought "Oh yeah this is going to be another Never Winter Nights 2" And guess what???
User avatar
Danger Mouse
 
Posts: 3393
Joined: Sat Oct 07, 2006 9:55 am

Post » Thu Feb 04, 2010 12:25 pm

This game is so very big, it's impossible to see every glitch, every error, etc...
I think they did a realy nice job for a game this big!


Yes, it's impossible to fix every glitch in a game so big, but they still could have managed to fix the big obvious ones.
User avatar
Siobhan Thompson
 
Posts: 3443
Joined: Sun Nov 12, 2006 10:40 am

Post » Thu Feb 04, 2010 9:08 am

I seriously doubt the game only had three testers, although it's possible the person was referring to the fact that the game must have been tested by three groups: Obsidian/Bethesda, Microsoft, and Sony.

From my understand for a game, DLC, or patch to be released on the XBox or PS it has to go through MS's/Sony's own QA process and be certified.


Actually.

I was reading the game's credits last night (textfile in the DATA directory) and I was stunned that in the QA listing there were only 3 people's names.

I was surprised to see this small number. With a game like this I guess I expected a QA department of like I don't know, 10? Those 3 people must work their asses off.

It seems that their coders and designers must do bunches of testing themselves because this game is way bigger than 3 testers, if you know what I mean.
User avatar
Catharine Krupinski
 
Posts: 3377
Joined: Sun Aug 12, 2007 3:39 pm

Post » Thu Feb 04, 2010 7:07 am



Ok, done ranting now. I tell you though, if we could return our software under the heading of "doesn't work as advertised", junk like this wouldn't hit the market because the cost of the returns would out weigh any potential profits.

Exactly. Were we allowed to return video games like we can any other flawed product then Bethesda would feel the hit. Unfortunately, video games seem to be under some sort of special protection. So Bethesda lives on to release broken products at a high price to unsuspecting consumers. It's sick.
User avatar
Chris Guerin
 
Posts: 3395
Joined: Thu May 10, 2007 2:44 pm

Post » Thu Feb 04, 2010 1:40 am

Exactly. Were we allowed to return video games like we can any other flawed product then Bethesda would feel the hit. Unfortunately, video games seem to be under some sort of special protection. So Bethesda lives on to release broken products at a high price to unsuspecting consumers. It's sick.


And yet you keep buying their games.

Guess that makes you a fool
User avatar
Kelly James
 
Posts: 3266
Joined: Wed Oct 04, 2006 7:33 pm

Post » Thu Feb 04, 2010 1:08 pm

I guess you haven't realized yet: you ARE beta testing this game.


People keep saying that, but it doesn't make much sense to me. For one thing, a company should take reports from the beta tester and patch the game accordingly. Is that happening? Didn't with FO3. Most of the so-called beta testers of New Vegas don't even contact Bethesda or Obsidian at all. That isn't beta testing, it's just people playing a buggy product.
User avatar
Alba Casas
 
Posts: 3478
Joined: Tue Dec 12, 2006 2:31 pm

Post » Thu Feb 04, 2010 3:28 am

I guess you haven't realized yet: you ARE beta testing this game.
It just wasn't free; you had to pay $60.


That is so funny but true! We are all paid beta testers... It is just that the pay went the wrong direction! ha ha ha.
User avatar
JLG
 
Posts: 3364
Joined: Fri Oct 19, 2007 7:42 pm

Post » Thu Feb 04, 2010 9:39 am

Exactly. Were we allowed to return video games like we can any other flawed product then Bethesda would feel the hit. Unfortunately, video games seem to be under some sort of special protection. So Bethesda lives on to release broken products at a high price to unsuspecting consumers. It's sick.


The law that they follow is a very out dated law that protects all publishers from typographical errors. There is no requirement to fix errors in books be they simple typing errors, omissions or other such nonsense. Software falls under these same publication laws. There needs to be a new set of laws created for software to cover at least 2 major points... Operates as intended and causes no harm to your system.

There are many programs that have major bugs in them, crash and lockup (not just games) and there are even some that when they go wrong will thrash your system (not on a console that I have heard of but on PCs). And the publisher is off the hook for any damages and they don't have to refund your money or even release a fix for free. They can (and one big company I know of) charge you for the next version that is reported to fix the bugs (but may not and may add more).

Big companies solve these problems through lawsuits but that doesn't help you or I.
User avatar
Stefanny Cardona
 
Posts: 3352
Joined: Tue Dec 19, 2006 8:08 pm

Post » Thu Feb 04, 2010 10:20 am

People keep saying that, but it doesn't make much sense to me. For one thing, a company should take reports from the beta tester and patch the game accordingly. Is that happening? Didn't with FO3. Most of the so-called beta testers of New Vegas don't even contact Bethesda or Obsidian at all. That isn't beta testing, it's just people playing a buggy product.


I agree. A real beta tester must have a way to report bugs through an official channel and they should be required to do so or not be allowed to beta test in the future. Also, they should get some sort of "time limited" version of the product. For a game, I would think that X number of hours of game play then the data files and game stop working. For more testing they would need to get a new copy of the game after various fixes had been applied.

That was the most common method we used back in the day when I use to be developing game software (way back in the Atari days) I still use that type of concept now that I develop business software. This keeps people from using beta versions of a program for production use. Normally, my beta testers get a free version of the finished product as payment for their services.
User avatar
James Rhead
 
Posts: 3474
Joined: Sat Jul 14, 2007 7:32 am

Post » Thu Feb 04, 2010 9:09 am

This game is so very big, it's impossible to see every glitch, every error, etc...
I think they did a realy nice job for a game this big!


This game wasn't tested, I don't think. That may seem impossible to push a title with any qa but I think thats what has happened here. If you want to read some of my constant problems then read my topic. The point is, that with cinimatics, and auto saves on in game I get about 20 odd minutes of gameplay before my console shuts down on me in a perma-freeze requiring either a hard reset, and if that fails an unplug from the wall to remedy. That is no joke, no exageration an odd 20 minutes or so. If I shut off all auto save features and cinimatics and try not to use VATs I extended my average 20 minute play time to about 3 hours before I hit the perma-freeze status. Also theres various other issues, continous quests you can turn in over and over, perks working sometimes then not other times. Quests bugging out to the point they cannot be accomplished, millions of game breakers threw out the world, resulting from the less serious fall threw the ground, and get stuck on some random item in the world till the most severe faction mess ups, where for whatever reason all quest givers of a given faction turn hostile to you regardless of your reputation with them. Other than that theres other issues i've come across, mobs stuck to eachother(that svcks when you got 2 people stuck together and you gotta kill one and save the other......) hacking issues, for instance certain gun sights get bugged and carry the hack terminal display for its scope, pretty much [censored] your game... Vats sometimes slowed down my game to the point I could barly move, even camera movement is very very slow, also in this... terrible terrible "your gonna die" mode, you cannot shoot, even with melee weapons you get the out of ammo click when pulling the trigger, gets better in that mode too, enemies are slowed but only about half as slow as you, and continue to barrage you with missles, dynamite, gunfire, laserfire or dog bites, until either the game corrects itself with you having about 5hp, or you just die. Also i get a VATs bug constantly that just goes in, plays for abit, i don't fire one round and goes back out, sometimes this takes my AP other times it does not.. regardless thats still many many seconds lost in a gunfight. Some models aren't put together really well, for instance the coyote if your centered and he leaps at you center, go into VATs to stop it, you can clearly see straight threw him, theres no textures in his mid chest section.. other graphical blips like this are threw out the whole of the game. I've also encountered enemies with hp that regenerate at a staggering rate making them hard to down. I call that bug the almost immortals. I've seen items fall threw the ground and i'm unable to reach them, I've also been killed or kill things threw walls doors, fences, ect. I've also killed things from below the ground at a fall threw point, was able to fast travel out of it, then on another fall threw point upon fast traveling i appear in the location but still under ground. I've had enemies also fall threw the ground and kill me without me being even able to target them(this includes VATs). Same bugs effect companions, i've had Eye bot fall half way in the ground, and get mawed by scorpions, I've had companions dissapear i assume fall threw the ground at some point and get the "blank has died" message an odd hour or so later.
Sometimes even out of VATs i hit a point where my gun doesn't want to shoot, a work around for that is to reload again(if you continue getting out of ammo clicks). well.... well well... Most of this was in my first day, dare i say my first few hours in game. So no... couldn't have been tested, I can't beilive that it was. I feel better thinking bethseda had no idea what they where shipping, than to beilive they intended on this being the product with their name stamped all over it. I agree with alot of the posts I see that this is a great game. I beilive under it all and its horrible horrible ending(Didn't learn your lesson?) it is a good game..
its just hard for me to vouche for it and say YA! Bethseda you rock fallout is Awsome! When i gotta sit in the floor near my console playing becuase I gotta hard reset the thing every half hour and keep 30 different saves.... Also beyond the game play, i've rigerously tested almost all perks and builds, seeing how it takes me about 20 minutes to have a character level 30, infinite caps/ sticks/ ect. becuase of the infinite turn in quests littered all over the world. It seems this game was meant to play as a melee character, don't beileive me, test the builds. You'll find melee characters 10x stronger than any other, even non melee characters do just about equal if not better with melee weapons. Even perks are geared more toward a melee character(theres also multiple special melee weapons in the world) That is also disheartening, Take a 10 agi, 100 gun character with a decent gun, and a 5 str 50 melee character and the melee > gun, same with energy weapons, same with explosives, it is more efficient, quicker, and easier to play the game as a melee character. Even possible perks show this, -Penetrating strikes, Unstoppable force, Purifier, Super Slam, Ninja, Slayer, ect. ect. goes on and on, there are also afew special melee moves you can learn from quests threw out the game(if it works). But anyway... starting to ramble....
User avatar
Penny Courture
 
Posts: 3438
Joined: Sat Dec 23, 2006 11:59 pm

Post » Thu Feb 04, 2010 1:27 am

And yet you keep buying their games.

Guess that makes you a fool




Well in my experience, i've played Daggerfall, Morrowind, Oblivion, Fallout 3. On all those titles I encountered bugs, but Sirs, i've never in all my days been fed a product such as this, Never.... ever... especially on a console. I've never even played a beta that gives you 20minutes-3 hours of play time prior to a crash. And I enjoyed all those titles, had alot of fun, I don't think I ever encountered any game breakers in those. And as such I trusted Bethsedas' name. Never read any reviews on it, it was a new fallout from bethseda, gonna be awsome, before I played even checked PSN to see if there was any add-ons yet... Thankfully there where none, or i'd bought them. That is how much I trusted the quality of the product i had purchased. I was wrong.
User avatar
Beat freak
 
Posts: 3403
Joined: Thu Dec 14, 2006 6:04 am

Post » Thu Feb 04, 2010 7:30 am

i agree with InTheMojave. Yeah, I've played lots of gamesass games. they're good games. yes, each of them was little buggy, but none were as buggy as this.
User avatar
Del Arte
 
Posts: 3543
Joined: Tue Aug 01, 2006 8:40 pm

Post » Thu Feb 04, 2010 5:58 am

Let's be realistic rather than hyperbolic: the game was tested. There are testers in the credits. Could they be lying about that? Of course. Is there any real reason to believe that they are lying? Not in my opinion.

The issue is how testing results are applied. My experience from viewing the industry as an educated consumer is that it is a business decision that causes situations like this to occur. Testers test, they report results. Producers and project leads prioritize the issues and determine which get fixed, which get partial/fast bandages and which get ignored. Developers don't get to call the shots on what they develop/fix.

It's all business.

We, as consumers, can do little else than vote with our wallets.
User avatar
Richard Dixon
 
Posts: 3461
Joined: Thu Jun 07, 2007 1:29 pm

Post » Thu Feb 04, 2010 12:30 am

wow... And I thought I could put on a rant...

Yeah, the game is awesome in concept and svcks like heck in practice due to all the bugs that could be found within a couple hours by anyone picking up a disk an playing. not testing this or that function or this or that section of the game in isolation. Pick up the game and play it and you will find bugs. No one I know of has said "I haven't seen any odd things in the game".
User avatar
Kelsey Hall
 
Posts: 3355
Joined: Sat Dec 16, 2006 8:10 pm

Post » Thu Feb 04, 2010 3:04 am

Let's be realistic rather than hyperbolic: the game was tested. There are testers in the credits. Could they be lying about that? Of course. Is there any real reason to believe that they are lying? Not in my opinion.

The issue is how testing results are applied. My experience from viewing the industry as an educated consumer is that it is a business decision that causes situations like this to occur. Testers test, they report results. Producers and project leads prioritize the issues and determine which get fixed, which get partial/fast bandages and which get ignored. Developers don't get to call the shots on what they develop/fix.

It's all business.

We, as consumers, can do little else than vote with our wallets.


It isn't "business" it is theft and fraud. The game is advertised to be functional and it is not.

The game was not tested by any reasonable standard.

Yes, they had testers and QA staff that most likely tested bits of the game in isolation but I will guarantee that no one took the time to play the game from start to end on a real PC, XBox or PS3 prior to the release date. Not one person. As so many people will point out, the number of variations and possible combinations involved in this game are staggering. It would be virtually impossible for a hand full of people to randomly reproduce the exact same criteria and encounter the same bugs without those bugs being easily detected by just about anyone.

Now, find me a person that has played this game from start to end on and XBox or PS3 or even a PC that has not encountered some of these bugs. If you can't find a single person using a real system then that is proof that this game either wasn't tested, or it was tested, they knew it didn't work and they sold it as "works great" when they knew it didn't work at all.

So which is it? They didn't test it? Or they tested it and sold it knowing it was badly damaged? It has to be one of those two.
User avatar
Kara Payne
 
Posts: 3415
Joined: Thu Oct 26, 2006 12:47 am

Next

Return to Fallout: New Vegas