Button-Mashing Monkeys

Testing unreleased games is both fascinating and mind-numbingly boring.

It’s a strange contradiction. Testers get a chance to see the development process unfold as new builds arrive fresh from the programmers. They are in a unique position to watch new games being built piece by piece, as bugs are fixed and new components added. If you’ve ever wanted to see a game evolve, testing is a good way to do it.

For some, it’s the ultimate dream job. Many aspiring game developers see testing as a way to get a foot in the door, a stepping stone on the career path to a “real” game development position. But others have a much more dismal view of game testing, envisioning it as little more than a room full of button-mashing monkeys. Even so, people continue to flock toward the game testing labs at Nintendo of America (NOA) and Microsoft Game Studios (MGS), eager to live their own small part of the game development dream.

Whether they’re climbing the development ladder or just there for the paycheck, however, they serve a vital role: Testers find bugs so players won’t. They are the unsung heroes of the game industry.

For Those About to Test, We Salute You
Most gamers are probably familiar with the open-beta model of testing, where eager consumers get to download a pre-release build of an upcoming game and test its multiplayer capabilities. It’s as much a marketing exercise as a development tool, a chance for fans to preview an upcoming game in return for providing some feedback.

Unfortunately, professional game testers rarely have the luxury of choosing what they test. Contractors at Nintendo and Microsoft are typically assigned to whatever titles are in the company’s test queue, and those assignments aren’t always interesting for everyone. After all, not all gamers enjoy the same types of games, and action game fans may find themselves testing a strategy game instead.

Even if you get to work on a game you enjoy, it can be tiring to play the same thing day after day for months on end. Test managers sometimes try to mix things up for variety, but the process of testing a game takes a long time. As a result, fatigue is a very real problem that even unlimited amounts of free coffee can’t solve. Testers nod off more frequently than might be expected in a tightly monitored environment. This is especially an issue for younger testers, who may be burning the candle at both ends trying to juggle an active social life with the rigors of a full-time job.

There are also some unusual side-effects that come from playing the same game for months. One quirk is that it’s almost impossible to play a game for that long without becoming exceedingly good at it. This can lead to awkward social situations once the game is released, with testers unable to play on equal footing with their friends. Likewise, some testers are simply no longer interested in playing games they’ve tested for fun. Galen Davis, formerly a tester at Nintendo for over a year, still hasn’t turned on his Wii since quitting his job three months ago. After spending thousands of hours in front of a Wii at work, his Xbox 360 is a much more refreshing entertainment option.

Surprisingly, there can also be health-related side-effects of being a tester. In a cramped work environment, germs and sickness are a problem. There is disinfectant available, but players don’t always use it when swapping controllers or equipment. In addition, because testing is a low-paying position, some testers who are strapped for cash may show up for work even if they’re not feeling well. Davis’ wife remembers him coming home sick a lot. Playing games may be good for your mental and physical health, but testing them can certainly exact a heavy toll.

Timing is Everything
When you’re just starting out, timing is everything. Entry-level testers are expected to show up promptly in the morning and work on a tightly maintained schedule. They arrive at the same time, take breaks at the same time and leave at the same time. For the novice contractor, everything runs like clockwork. “At Nintendo it was 7:30 to 5:00 with two 15-minute breaks and an hour lunch,” remembers Derek DeHoogh. Having recently made the transition to Microsoft, DeHoogh notes that the hours are marginally better, but not by much. “Now I work 9:00 to 5:30 with a 45-minute lunch and a 15-minute break.”

Timing can also be an issue when it comes to multiplayer. At MGS, there are pre-arranged times when testers are asked to go online. This is as much about testing the hardware and the infrastructure as it is about testing the games. Gamers expect the online components to work out of the box, and testers are there to make sure that the games and hardware live up to those expectations.

From a corporate standpoint, quality assurance adheres to the same schedule as the rest of the development process. To speed things along, testers are often given savegames from various stages in the game. This usually includes access to the unlockable content, removing both the hassle and the joy of unlocking it through regular play. Of course, testers have to check to make sure that they can unlock that stuff normally, but that part gets looked at later.

Dealing with a company based overseas introduces its own set of time-zone-related quirks for the testers at NOA; Davis remembers going in at 4 A.M. for some special testing that required talking to the Japanese development team. This is one area where the Microsoft testing experience is more straightforward, since the game programmers are often just a shuttle ride away.

Movin’ On Up
Like Nintendo, Microsoft also employs a lot of younger, entry-level contractors as testers. For both studios, the daily testing routine is very similar, with regular hours and assigned projects. However, testers at Microsoft are paid a bit better than at Nintendo, which has some interesting implications. One result is that testers tend to stay around a little longer. This opens up new opportunities for experienced testers to advance up the career ladder.

At Microsoft, there are opportunities for game testers to advance beyond their starting contractor position. Senior testers can earn the title of Software Test Engineer, and with the title comes more pay and better hours. “When I come into work varies on what needs done when,” says Robert Lamb, a STE at Microsoft. “There is a lot of freedom of when you get in, provided you get what you need to get done, done.”

After putting in some time at the senior tester level, skilled contractors can work their way up to full-time employment with Microsoft, with a regular salary and full benefits. Of course, some make the jump straightaway. “If they do skip the contracting stage, it usually means they have coding experience, and come in as an SDET,” says Brian Fetty, another STE. That’s a “Software Design Engineer in Test,” a programmer charged with writing test automation software tools which take care of the more mundane aspects of putting a game through its paces. Automation is like having testers that can check for stability issues 24 hours a day, 7 days a week without getting drowsy or sick.

Of course, automated software can only test certain things, and there is no substitute for the creativity and judgment of a human tester. Besides, an automated system can’t tell you whether or not a game is fun to play. For that, MGS employs separate usability testing groups, sometimes very early in the game’s development, to test out new ideas. Other usability tests take place much later in the development cycle, with outside playtesters who can examine a game from a fresh perspective. “These guys are the ones living the popular conception of game testing; play games for a living, get your chance to let the developers know what would be great, and what isn’t working,” says Lamb.

At Nintendo, contract testers can also report problems with gameplay, although it isn’t clear whether or not the developers do anything with those reports. Feedback on the playability of a game is reserved for a select group of full-time NOA employees, or so the rumor goes. Davis wasn’t exactly sure; Nintendo is deliberately tight-lipped about the subject. But some of the Nintendo perks are not as closely guarded, like the ability to shop at the heavily discounted company store. For a gaming enthusiast working as a tester, perks like this are part of what make the whole experience worthwhile.

Putting It All Together
So which is it: “dream job” or “button-mashing monkey”? At Nintendo and Microsoft, testing is a little bit of both, and the perks of getting to play unreleased games must be measured against the drawbacks of playing unfinished, potentially broken builds for days, weeks, and months. Game industry jobs are highly sought after, even for experienced testers coming from the business software world. As with any other job, however, the corporate culture makes a difference.

At Nintendo, some of it has to do with their traditional corporate structure. Company-wide directives work their way down from the top management in Japan to contract testers, who are given strict work schedules and testing assignments. Nintendo is all about raw manpower; testers are a valuable resource in ensuring that a game gets enough time in front of human eyes and hands.

Entry-level contractors at Microsoft share an experience similar to that of their Nintendo counterparts, but the paths diverge for senior testers. Nintendo moves their senior testers into management roles, while senior Microsoft testers move up in the engineering career ladder. Experienced testers are tasked with the development of automated testing tools to handle the more repetitive aspects of game testing. For them, testing is a technical problem to be solved through engineering rather than longer hours in front of a console.

From a career perspective, DeHoogh tells me “it’s pretty common practice to do a year or so at Nintendo and then move up to Microsoft. They do pay more and Microsoft takes care of its employees more.” Plus, he mentions that the food is better.

Some contract testers eventually leave altogether for greener pastures, giving up the thrill of game testing for a larger paycheck and the benefits that come with a full-time position. Others stay and advance along the tester career track. After all, game testing has its own set of unique perks. Despite the low pay and the repetitive work, there is still something glorious about the opportunity to watch games grow and change while getting paid to play them. As Lamb puts it, “After six years I’m still dreamy eyed over making a living while also making games, and a large bank account never motivated me in anything.”

Alan Au is a freelance writer, academic, and games industry advocate. When he’s not trying to circumvent bugs that the testers missed, he spends his time exploring the connection between games, education and health.