First Impressions: Storm of Zehir

I just fired up the latest official expansion to Neverwinter Nights 2: Storm of Zehir. I put about an hour into it and walked away unimpressed. I think I might not be in the right mindset for this. I’m going to come back in a week (after the 1.24 patch is released) and try again. In the meanwhile, here are the first impressions I jotted down as I played:

Told to scavenge shrubbery and wreckage after the ship crashes. I’m off doing that and find out I missed a battle. I barely had time to get situated. Should have made the first encounter a player-triggered encounter – that is, the monsters wait for Volo to approach, and you tell Volo when to approach. That way you can prepare yourself.

Player should have the option to speak with the monsters. I put a lot of points into dialogue and would at least like a chance, even if Volo is better. Let the player play.

Volo says he’ll talk to the natives but he just stands there. If I get too close he tells me to back up. Nothing happens here, so the player is forced to walk close enough that it triggers the monsters to go hostile.

A patrol arrives and tell me I’m under arrest but, again, nothing happens. I wandered around for a while and then eventually spoke with the patrol, at which point the game continued. If the goal is to give the player a chance to finish looting, perhaps the guards could have told us that they were giving us a chance to gather our belongings as a matter of civility before dragging us along. It would have felt less broken.

Dialogues when I scavenge for items are unclear. Feedback kept telling me I found an item but I didn’t think to look to the chat window to see what I had gained. I was expecting that information in the dialogue window where the game was telling me I found something. If it’s just the way the system works, then there should have been some hint — check your inventory, etc.

Multi-player dialogue interface is unclear. Every time I saw the speech bubble icon on a player tab, I’d click on those different character tabs to see their bonus dialogue options. But then I’d have to check back against the other character to determine which options were the new ones or unique ones. They should have all been listed on one page with a character name/names prefacing the line so you’d know which were the unique options.

In other dialogues, I was having trouble following along and figuring out who was saying what. Made me a little frustrated and made me feel like just skipping through the dialogue instead of immersing myself into the story.

I created a new character but she never was gifted the necessary XP to level-up to 4th like the others. Had to create an entirely new character.

Related, the game never tells you what level of character is appropriate to start the game with. I was initially tempted to drag over my character from Mask of the Betrayer. He’s close to level 30, I believe.

I clicked on the door to “belowdeck” and was given a confirmation screen (do you really want to start the adventure?). I accidentally clicked on it again later but there was no confirmation this time. I needed the confirmation as I was not yet done.

Familiar doesn’t follow. My cleric isn’t always healing like he should. My wizard used up all the charges on a wand in the first wave of the first encounter against the monsters with Volo.

Made it to the overland map and am giving that one a go for the first time. Wow, monsters spawning like crazy right around the town. Like, every few seconds new monsters are popping up. They’re everywhere and charging me constantly. And all different kinds — gnolls, orcs, zombies, ogres, and skeletons in the first 30 seconds of travel. This isn’t a JRPG! My party is getting decimated.

I saw a patrol fighting zombies. I decided to intervene but there was no dialogue or anything. Everything suddenly felt very generic and it made me not want to bother with future encounters. I was spotted by another group and led them in a circle until a patrol picked them up and I could continue. That felt very dumb during/after the fact.

Spotted a few typos and grammar errors but that isn’t even worth mentioning in light of the above.

Saints Row – Bugs

I just cleared the 20-hour mark in Saints Row. And what stands out to me the most? It’s a fun game. I’m really enjoying it and I always look forward to playing. That said, bugs are a major nuisance – so I’m taking a moment to comment on the first seven issues I would have bugged if I were on their QA team:

  1. The Main Menu defaults to New Game, not Continue. When you click New Game, you jump to a loading screen which then takes you to character creation. If you wanted to Continue but accidentally hit New Game, it’s going to take you a little while to get back to the menu. If the game defaults to Continue, then all it does is bring up the available saves. No load time necessary. And most likely, that’s what you want to do.
  2. Audio volume levels start at maximum, then shift to the user settings. Every time I jump into a car, start up a game, go through a cutscene, etc. – the radio kicks in at full volume before dipping down. Annoying.
  3. There’s some strange stuff happening with the car physics, but it’s usually nothing too bothersome. Sometimes I drive under or over a car. Sometimes I hit a car head-on and we both come to an instant stop – with no damage or collision impact. Sometimes my car cannot drive over a short curb. Etc. Wonky as they may be, sometimes it makes for interesting happenings: like driving over a car and making it explode, resulting in my car soaring through the air. I launched over a bridge and crashed into the sea. Fun!
  4. The x/y-axis for the controller [LS] seems a little tight. That results in the character occasionally walking in a direction even though I’m not even touching the controller. This happens a lot and can be irritating. I’ve actually been killed a few times because of it. For example, I’ll attempt to dive into a car to get away from trouble. I hit Y to enter, my player races to the door, opens it, and then slowly walks away. This happened three times in a row the other night!
  5. There have been a lot of graphic glitches. Sometimes they scare me. I’ll be driving along and my car will disappear or the road will disappear. Other times, it’s just the standard “popping” where it jumps from low-res to high-res textures.
  6. Pathfinding is generally bad. During one mission, I tried to lead my gang up a set of stairs and past some metal railings. They weren’t able to handle that and I actually had to take a round-about way to get my gang to the proper location. Another time one of my gang members couldn’t get past an open car door and I had to jump in-and-out of the car a few times before she could successfully leave the car. Behavior like this is fairly normal for the game.
  7. As you play, you claim neighborhoods as income-generating territories. Occasionally, a rival gang attempts to steal one of your territories, forcing you to kill a certain number of rival gang leaders. Unfortunately, the leaders are often placed right on the border of the territory or just outside it, or placed in a way that you have to leave the territory to get to their location. Every time you step outside of the territory you get a warning telling you that you have 30 seconds to return to the combat zone. There really should have been some effort made to center the gang leaders within the territory.

I’d have been tempted to file “inconsistent difficulty” as a bug, but that’s sort of an accepted issue when dealing with games of this nature where anything can and will happen. In other words, some missions seem impossible because of how events proceed. But after a dozen tries, everything works in your favor and you get through the mission without a sweat.

For example, I came upon a combat zone and was having difficulty getting past it. After a series of tries, I accidentally struck a critical area with a rocket, as I was approaching the area – resulting in a series of chain reaction explosions that wiped out the entire resistance. Go figure.

The game is buggy and it does require some patience, but fortunately it’s also very fun and that compels me to overlook the issues as I play. Maybe for the next post I’ll update my impressions on the game as a whole.

[edit #1] I had an awesome sequence just now. I think it was the last West-Side Ballerz mission. I had to chase some guy named Prince on a freeway/bridge. He was driving away in a car-carrying truck. While he throws bombs at you and random goons hassle you, you have to shoot cars off the truck to damage the truck. Once you dislodge him from the carrier, you chase the cab down and blow him up.

I tried and failed maybe 3 or 4 times, but despite the difficulty and bad frame rate I was enjoying the challenge. But then, on my next effort, the truck’s cab jammed into the ground, forcing the carrier behind it into a vertical position – kind of like a mech. And it was stuck. So instead of a tough driving sequence, I just had to circle around the truck, fighting goons and taking shots at the attached cars. Once the truck had sustained enough damage that the cab dislodged, it fell down through the bridge. Because I was on the freeway, there was no way for me to go to it. I couldn’t jump down to the road below without dying. I couldn’t drive back to the connecting road without abandoning the mission.

After killing dozens of hassling goons, I spotted that the freeway has supports on its side where I could stand and stay away from the goons. I figured maybe I could line up some shots from there on the cab. I hopped over the freeway side and it worked perfectly – except, the map where the cab was had never drawn in! It was just empty black space. I could hear the cab but I could not see it. I waited for a moment and then a screen popped up – success! I won the mission! Hahaha. Go figure.

Buggy!

[edit #2] During the mission “The King and I”, you’re told to lead three separate groups into an ambush. It’s pretty simple. You drive up to the group, shoot at them or something to get their attention, they hop into a car, and then you drive just fast enough to keep them close to you while leading them to the target location.

The mission was going fine until I hit the third group. I got to the location and waited. And waited. And waited. I had lots of goons to fight but the car that was supposed to follow – wasn’t following. Finally, I got out of my car and looked around. I found their car stashed nearby beneath a bridge, and a group of two running away from their car. But! They were running away from the car and into the underside of the bridge. Or, in other words, it looked like they were stuck.

Fortunately, the group was tagged as immortal. So to resolve the situation, I ran back to my car, hopped in, lined myself up with the two guys, and then plowed into them full speed. It launched them away from the bridge area and seemed to re-set them, as they then ran back to the car and properly hopped inside to let me complete the mission.

[edit #3] I was racing back to home to complete a mission and I did a power slide through a turn a little too late. The right-side of my car smashed into a brick fence. One of my passengers vanished, as did the car.  My character and the two remaining passengers were all posed as if they were sitting in the car. They even occasionally stood up as if they were firing over the roof of the car. At one point, a police officer ran into the dead-center of where the car would be, standing between the driver and the front-seat passenger, before getting shot by the back-seat passenger. I waited for about 30-to-45 seconds before deciding to re-load the mission – when the car popped back into existence, along with the missing passenger, and we were going full speed as if nothing out of the ordinary had ever happened.

Quality Assurance Analyst

When I worked in Quality Assurance and people asked what I did, saying “QA” returned only blank stares. Even saying “Quality Assurance” is little better for those who have never played video games. But if you say “I’m a tester at a video game developer” then there’s instant recognition: “Oh, you play video games for a living. Hahaha! Life must be so tough! Lucky you!”

This perception infiltrates through the game-development community and inter-office as well. Throughout the game-development industry, the title “QA” (often mistakenly listed as “Q&A” – my job is not “Questions & Answers”) signifies exactly what the perceptions are: high-school students working part time for a low wage, tasked with the responsibility of playing through a game for a month or two and, hopefully, articulating, at least at a very basic “yes/no” level, whether or not they had “fun”. These employees generally work for the publisher (as opposed to the developer) in the time period around the game’s release, and are hoping to parlay this light introduction to development into an “actual job” – as a writer or designer or programmer or producer, etc.

A few companies, developers and publishers alike, BioWare being one, employ more specialist teams of QA. At BioWare, we were called QA Analysts. It’s not really much of a separation in terms of name, but the roles and responsibilities form the casual tester are quite different – even if the Analyst is still required to “play the game”.

The key differences are that the Analyst is paired with developers and that the Analyst is there throughout the development cycle. And the Analyst isn’t just playing the game. Instead, the Analyst is responsible for ensuring every facet of the game is the best that it can be, while simultaneously understanding limitations in time, scope, and budget (the staff).

So how does an Analyst handle such an immense task? Well, first of all, most projects employ a streamlined, focused team propped up by contract QA. From there, the game is divided into parts to simplify roles.

For example, at BioWare we split the process into two branches – with one side focused on the game and one side focused on the systems.

A System Analyst would be responsible for, amongst many other items, the duty of testing animations; to perform this job, the Analyst might create a specific “dummy level” to allow him to quickly ensure behavior works properly in all situations.

One level I looked at often for Mass Effect was a large room with every monster type standing at ready. Since there were no scripts running (AI) and the level was just a plain room, it wasn’t taxing on the system to have so many creatures in one place. A lever in the room allowed me to make every creature cycle through their animations – from walking (in place) to running to dodging to attacking, and so forth. In this way, it was very easy for me to look over every creature in the game and quickly report any failed animation cycles. Ultimately, many levels are designed to aid in quick testing and a System Analyst will spend most of their time working with programmers.

A Game Analyst looks at created content filtered through the context of the game experience.

If a creature plays a broken animation and the system test shows that all animations are working, then the Game Analyst knows something is going wrong inside the game – perhaps with scripting. Maybe the game is telling the creature to play an animation that it doesn’t have; maybe the game is telling it to move at a rate it cannot process; maybe something else is acting on the creature to prevent it from properly playing its animation. It is the Analyst’s job to figure out what that bug is or, at least, find a sure-fire way to reproduce it so that the designer can figure it out. Regardless, the Game Analyst focuses on everything that happens while playing and works primarily with designers and writers.

An Analyst, either branch, usually has a few “testers” or “contract QA” or “junior QA” assigned to him – who also do more than just “play the game”. These testers take directed routes through the game and look for specific issues.

For example, a game tester might be told to play through one specific level as a female character using a pistol and look for balance issues (judgments on game difficulty revolving around using a “weaker” pistol versus some other more powerful, preferred attack), while a system tester might be tasked with opening a specific level and looking for bad walk mesh (places where the character can “escape” the level and break the game – and, again, specific tools are usually created to aid in this process, such as an exploding “paint” effect to help locate “seams” in the mesh).

Side note: although personal judgment plays a large role in testing, the goal is always to keep things as objective as possible. Research and test plans are great aids here.

The testers report any issues that they find to the Analyst, who is responsible for managing the bug database for his assigned level. And further up the chain of command we have the Tech Leads who are responsible for the entirety of the game or systems and their respective databases. Finally, we have the lead who is responsible for managing the entire process and, in BioWare’s case, communicating directly with the Producer.

But even this isn’t the entire picture, because it still implies a reactive testing (that is, developer creates finished content, delivers it to QA, and then the content goes through the test/fix/approved/regressed process) which is, really, an expensive and problematic model to rely on for all testing.

Thanks in part to the introduction of “agile” development, QA is now more often employed from the very start. It is still “reactive” (and QA is a service department so that’s their intended role) in the sense that a writer might put a story together and deliver it to QA for criticism and feedback, but by providing feedback through the various milestones the cost of changes are much cheaper.

That is, if you decide to change the story when the game is complete, even a small change could mean throwing out tons of scripts, character models, animations, voice acting, and so forth. A small change could effectively force a dozen people or more to spend weeks or months of time to implement the “fix”, and then weeks or months more to test the specific changes as well as everything that was touched by developers in the process. On the other hand, a drastic change when the story only exists on paper means one writer going back to the drawing board for a few hours or days.

In other words, do you really want to put QA on a project a month before the game’s release only to hear them tell you “this isn’t very fun”? No, you’d rather them let you know the game’s weaknesses when everything is in prototype and can be easily changed without impacting the schedule.

In this way, an experienced analyst can drive development, saying things like “we had difficulty testing this system on our last game because of how it was designed. I’d like to see it designed with this in mind to accommodate” or “our research shows that 23% of our customers never found this quest, and your new quest idea is set up in a similar manner. We need to come up with a solution to make it more visible if we’re going to invest so much time into creating the quest.”

As evidenced by these two examples, and the preceding comments, it should be clear that QA is not just “playing the game”. In fact, with so much going on, it is definitely possible for an Analyst to go days and weeks without even getting to play the game – at all, much less for enjoyment. Of course, while this does happen, you want to do everything you can to prevent this, up to and including working overtime. 🙂 Even as a Tech Lead, or my supervisor the lead, it was important to make sure we played through the game at least once a week, if not more, to stay current on the big picture.

While viewing the game drives knowledge and needs to be performed regularly, QA is about a complete involvement in the development process and seeing things from a bigger picture than many developers have access to. Looking for bugs is important, critically important, but it’s understood that QA has a lot of early benefit to offer also. Once everything is done, and solidly put together for beta, then everyone — QA, programmers, artists, etc. — can work together to find and fix bugs.

It takes a lot of study and experience, but it’s enjoyable and, I think, a very exciting career path – more exciting to me than many other available development opportunities. I understand this post does not cover many of the day-to-day details that arise in QA, but I hope it does a fair job at touching on the basics. Questions, comments, and suggestions are welcome.

Feedback Poll

Short Version: You’ve installed a game. You see a pop-up that reads “Would you like to volunteer for anonymous usage feedback in order to help us make better games?” Are you offended about the invasion? Would you volunteer? Respond in the comments and/or the poll in the right sidebar.

Long Version:

(today’s post: 250 words)

When I was at BioWare, Microsoft had a program that tracked everything a gamer did while playing. It could then generate reports on anything you wanted to know – such as how often people pressed the “A” button, how long people spent on the inventory screen, and which direction they turned at a particular crossroad.

As anyone who read my recent Facebook 2.0 post might guess, I (and more importantly BioWare) was thrilled about this discovery. Further, I remember mentioning how exciting it would be to gather all this data from our customer base. And that was when I found out that the program is stripped from games before they are shipped; the tool was intended to assist with internal testing only.

I begged for its inclusion in the final shipped game (I really wanted to know how many people beat our games, which optional missions they started, what their favorite talents were, which characters they talked to more often, which dialogue choices they more often chose, etc.) but I was told there were too many privacy concerns.

I countered by suggesting a pop-up at the game’s start asking if the customer would like to volunteer to help make BioWare titles better by submitting anonymous information during gameplay. No go.

So now, for curiosity’s sake alone, I turn it over to you. If you played a game and saw this pop-up, or were planning to buy a game and heard about this program, would it affect your decision to play/purchase the title?

LBP, meet QA

(today’s post: 250 words)

Lisa, Candice, and I have been playing through LBP (LittleBIGPlanet) for a while now and having a great time. It’s a great game. Maybe four days ago we made it to a level called “The Bunker” (2nd from the end!) and, after many tries over the course of a couple days, we beat it. Then… nothing. No last level.

Did we miss something? Candice scours the ‘net and finds nothing. I also find nothing. The level should automatically unlock when we beat the Bunker. I post our problem on the LittleBIGPlanet forums. No one knows what’s wrong. It would appear we have a show-stopper.

I can understand and accept that if I do something weird and unexpected, I’ll witness something bad (we actually fell out of one of the levels at one point, but we were being weird and unexpected). After seven years at BioWare, I’m fully aware that most developers take short-cuts and most games aren’t bug free or close to it (I’m educated-ly-guessing most shipped games have hundreds or thousands of bugs).

But this is a critical path bug. In a popular title. It does surprise me. We can’t finish LBP because something, I’m guessing, is blocking the save of a level’s state (you’d think that’d be a huge problem – but we’re one of the very few who’ve posted on the internet about this problem, so I guess we did something rare. One person at the LBP forums suggested it might have been a clothing choice.).

QA?! 🙂

Console Patching

(today’s post: 245 words)

I spotted the announcement of an upcoming patch for Fable 2 on the Xbox 360 and it got me thinking…

One of the perks of console gaming used to be that games were “finished”. Once the internet became prevalent, games on the PC were often shipped with bugs that were later fixed while console games had to work; there was no way to patch them. It forced console game-developers to have quality standards.

Starting with the original Xbox that shifted. The internet came to consoles.

At first, there was resistance. In fact, I remember working for a particular video game company and us sneaking bug fixes out to consumers, hiding a patch behind bonus content (like slipping dog pills in a piece of bread). I don’t know why it couldn’t just be said, “Hey, we know this is broken. Don’t do [this] to avoid the bug or download this fix.”

Maybe it was because everyone realized it’s a slippery slope.

Fable 2, in PC-like fashion, had a patch available day one (the day the game arrived in stores). There’s a second patch set to arrive in mid-December.

To be fair, the first Fable 2 patch enabled multiplayer. And I like bonus-content patches (even if there are bug fixes hidden in there). I also don’t mind rare issues getting addressed – like the bug on the aforementioned project I was on. But the idea of patches becoming common place is… well, it allows developers to be sloppy.

Where’s the quality assurance?

LittleBIGPlanet

(today’s post: 350 words)

Shortly after the release of Fable and Black & White 2, Lionhead artist Mark Healey left the company to co-found a new company – transitioning his Kung Fu puppet game along the way to what is now known as LBP (LittleBIGPlanet). I’d been excited to play this game (and the original Kung Fu title) but it wasn’t until Lisa and Jared bought a PS3 and LBP that I realized I’d get the chance.

Why was I excited? Curiosity! I thought LBP was a game that needed four people (online, in person, or a combination thereof) to play and the game revolved around getting through platformer levels as a group that had to rely upon one another; I really wanted to see how that could work since all I could think of were the technical obstacles.

Anyone who knows the game knows my assumptions were wrong. LBP is technically a single-player platformer that can be played with up-to-four simultaneously. Players have no need to interact with one another except during the occasional two-player bonus areas that aren’t essential to completing the game (though there is the occasional racing level where you can compete directly).

Don’t get me wrong. This is a great game. It’s as well done as Super Mario Galaxy. It’s quirky and colorful (the textures are amazing), never boring, and well-balanced (difficult at times but never beyond what you’re capable of). And when you discover, at 3am, that you can hit your friends with a frying pan, it’s pretty much the most hysterical thing ever.

Sackboy

(LBP’s customizable Sackboy character…)

But everything that LBP does so well makes me wish there were some teamwork dynamics – like one sackboy helping another reach a higher point, and then that sackboy reaching down to help pull the other sackboy up. Or maybe one sackboy reaches out and grabs hold of a sackboy who tried to make a long jump and came up just short.

As I said at the start, a game like that only makes me think of technical hurdles. But now I really want to see someone make that effort. Hopefully LBP 2 takes that step.