The Perception Engineers

It’s no secret that every product on every shelf in every store is subjected to a wealth of market research and end-user evaluation, but a little digging around in the Microsoft Game Studios earth uncovered an entire department of engineers and – what’s more unexpected – psychologists, all dedicated to quantifying the appeal and fixing the flaws in the human element of videogames.

Daniel Gunn is a User Researcher at Microsoft’s Games User Research Group, who tells me about the underlying function of his profession. “I’m trained in Human Factors Psychology,” he says. “When most people think of psychologists, they typically think of clinical psychologists who treat people’s phobias, personality disorders, depression and so on.

“Human factors psychologists, on the other hand, deal with how humans interact with the environment, tools, technology, information, etc. We strive to ensure that systems are safe, intuitive, productive and easy to use for the user; we focus design on the human factor. Most human factors psychologists have extensive training in human cognition, memory and perception. In addition, we typically have very strong backgrounds in experimental design and statistics.”

Despite being trained for such a multifarious profession, it seems a little hard to figure how someone like Dan fits into the grand scheme of making videogames fun and playable. His colleague, and another User Researcher in the Group, Tim Nichols shed some light on an engineering psychologist’s role in game development.

“Engineering psychologists are interested in the capabilities and limitations of users, and how best to design for those capabilities and limitations,” he begins. “Designers are always curious about how gamers interpret the game. Do they solve the puzzle the way they’re supposed to, do they get lost in the city, do they use the right weapon against the boss, do they notice the box of unlit torches in the dark cave (looking in your direction, Oblivion tutorial!)?” laughs Tim, clearly someone who is not only an integral member of the industry, but also a dedicated gamer. He continues:

“At Microsoft Game Studios, game designers can turn to the Games User Research Group to help answer these questions. Folks in my group have extensive training in how people perceive, how they interpret what they perceive and how they make decisions based on these interpretations. Combine that with our passion for gaming, and, basically, we’re very good at measuring how gamers react to games.”

Speaking to Dan and Tim has made me wonder about my own introduction to Microsoft’s current console. I can honestly say the Xbox 360, the PlayStation 3 nor the new Nintendo machine (whose name we shall not speak) held any interest for me. Eventually succumbing to a few test games in a local videogame store prompted an unexpected impulse purchase, one I didn’t relish having to explain to my accountant (who is also my wife).

After a few hours of obligatory resentment, she reluctantly gave the new console half her attention and has since worn away her fingerprints on the controller. Not only that, but my house has since become populated with exactly the kind of people you wouldn’t expect to be spending their time on videogames; my father, my wife’s friends, the nutcase kids from next door – none of them have ever been interested in this form of entertainment before. So why now?

I had to wonder how much of this unsolicited fascination was the result of a psychological prowess infused into a game’s early development.

A previous report from the User Research Group highlights exactly how Tim and Dan’s team deals with apparently minor aspects of a game (such as the user interface) to ensure even new players don’t start off with a negative impression.

During tests of Combat Flight Simulator, the Group flagged a usability problem that anyone other than a trained psychologist might easily dismiss as nothing more than a niggle. One of the selections within a setup menu was a choice of three radio buttons for adjusting the A.I. level of computer-controlled enemy pilots. Although it was quite reasonably assumed most people would be familiar with the term “A.I.” as an acronym for “artificial intelligence,” an early study showed a combination of trifling factors conspired to confuse the users.

The main problem was with the term itself. Although the participants had been selected due to their gaming and flight sim experience, the term “A.I.” was apparently not as well known as the development team had assumed. It may be common jargon for developers, but only two out of seven testers were actually familiar with it.

Had this minor problem not been highlighted, there was a danger novice users would begin their first few games of Combat Flight Simulator against an incredibly difficult enemy and scrap the entire game before getting to grips with it.

Dan confirmed this preemptive “tweaking” of a game (and its interface system) is what the Group is all about. “The idea behind the Games User Research Group is very simple: Collect unbiased data from real users during game development and use that data to make improvements in the game before it’s released. Although the idea is a simple one, the process is far from simple.” He laughs, summarily making light of his obvious hard graft. He continues:

“It’s not as easy as just bringing in the target consumers and getting their feedback. Careful control and expertise in psychological research methodologies must be leveraged in order to ensure the information we get from users is unbiased. The nature of how the testing is set up, the interactions participants have with the experimenter and other participants all have to be carefully controlled in order to ensure the integrity of the data we collect.

“The reason our group consists of individuals with a strong background in psychology is that we are specifically trained to gather data from people in an unbiased fashion, and we are well aware of all the potential areas for bias to creep in and work diligently to minimize that possibility.”

You’d think a team of psychologists searching and digging to find fundamental flaws in a game’s design would make them pretty unpopular with programmers, designers and artists, and Dan can recall a time when the Games User Research Group certainly had to prove its worth to the rest of Microsoft Game Studios.

“Eight years ago, when the group was first forming, many development teams questioned the benefit of having psychologists conducting user research on their titles,” Dan remembers, “but our group has grown substantially over the past few years, which speaks to the importance Microsoft places on the work we do.

“The most progress has been made in how teams approach making games. User Research is now seen as a key ingredient in game development, and the importance of our work is recognized by the development teams. Today, it’s actively sought after by the teams.”

This idea of a more collaborative effort between the User Research Group and the game designers proved invaluable during the development of Halo: Combat Evolved. The development team worked hard to make the game consistently fun throughout (no small task in such a huge, free-roaming game), though it turned out to only be fun if the users played as the designers intended. Which, of course, they didn’t.

The Group’s solution to this problem deviated from normal error counts and pass/fail criteria by bringing the actual game developers into the procedure to simply observe the tactics employed by users. The game had been designed with the assumption that players would be combating enemies in close quarters, but the shrewd users immediately discovered the targeting system allowed them to remain at a distance and pick off the enemies from long range.

Not only did this circumnavigate the deliberately designed “fun” aspects the developers had worked so hard to create, it also left the players bored and frustrated as they never actually saw the combat and felt their weapons were highly inaccurate from being used at such a long distance.

This time around, it was effectively the developers who were undergoing evaluation as they observed an unquestioned usability group. Their task became finding ways to encourage users to play as intended without forcing decisions upon them. This was achieved by adjusting enemy intelligence to dodge shots taken from long range and by advancing on the player to a position within the intended “fun zone.” The targeting system was also adjusted to have a range limitation, thereby influencing users to discover the enjoyment of close-quarters combat.

Spending their days examining the workings of a gamer’s brain provides the Group with a unique insight into the behavior and habits of those of us who dabble in the electronic arts; most of which we don’t even realize ourselves. Tim provided me with an inside glance at the psychologist’s perspective: “It’s always interesting to observe gamers perseverate on a usability problem in a game build. So often, gamers will bang their heads against a wall repeatedly, trying to figure something out. This may be different from non-entertainment software, because gamers have an expectation (learned over time spent playing many games) that difficult tasks in a game environment are part of the game, that these tasks have findable solutions and it will be fun to finally uncover that solution.”

And who among us can say they’ve never actually found themselves in exactly this kind of scenario?

The “power up” is almost reachable if I can just balance on this minute piece of scenery long enough to do a double jump in exactly the right place … damn it!

I’m happy to hold up my hand and admit I’ve jumped, double jumped and thrown away life upon life trying to get that damn power up, only to succeed in attaining severe thumb cramps and a bitter taste of defeat.

The difference between a difficult, yet deliberate puzzle and an unsolvable discontinuity in gameplay is not something I have ever really considered, but the process of trying to ensure our perceptions are guided in the right direction is an art form in itself, one best suited not to a game tester or a programmer, but someone trained to understand the volatile inconsistencies of human nature, just as Tim explains: “When the difficult task is intentionally implemented by the game designer, this is the case: There is a challenge, and it’s fun to overcome the challenge. (For example, think of the time when it finally all clicked in your head that one particular weapon used in one particular way would make a boss fight much easier.)

“Unfortunately, when there is an unintentional difficulty in the game (a ‘usability’ problem), there may not be a ‘eureka moment’; the problem might just be really, really hard to solve. In these cases, users sometimes continue to bang their head against the wall, trying to ‘solve’ a usability problem. The end result is that the user will eventually look up from their bleeding fingers and mangled controller, realize that the game is not fun and stop playing – even if they managed to solve the problem.

“We want to avoid these experiences at all costs. Psychology is a behavioral science, and while there’s a certain degree of peering into the black box, it’s definitely not a crapshoot when trying to figure out why people behave the way they do.”

Although they work this closely with designers and development teams, both Tim and Dan spend a lot of time with individuals outside of Microsoft’s user research program. Inevitably, these testers are also people keen to get involved in the dynamic area of engineering psychology and the gaming delights it brings. But, as Tim explains, a long history of videogame playing isn’t really what their bosses at Microsoft are looking for.

“I have friends with similar educational backgrounds who are testing how users interact with copiers or microwaves. Now, I have nothing against microwaves (Hot Pockets are a nutritional part of any meal), but I get to do the same work, only with space marines and RPG skill trees and awesome graphics and technology. This field is just fun to work in!” he says. “And inevitably, when I walk a usability participant to the exit, I’ll be asked about job opportunities. I don’t blame them; I’d want to do my job, too.”

Dan has an equal passion for his job:

“If I wasn’t working in the Games User Research Group,” he told me, “I’d be dreaming about having this job! I’d be doing something that involved helping to make some sort of technology or device more user friendly, but it definitely wouldn’t be as cool as games!”

I may not get to use my Xbox 360 as much as I’d like (or at all) due to the long line of “the recently converted” sampling their first taste of videogame enchantment, but it’s certainly a testament to the hard work of the people at the Games User Research Group the way they wait on my doorstep and huddle around my TV.

Fun, it seems, is a serious business, and it’s reassuring (especially for those of us who stretched our bank account to its elastic limits in order to join the next generation community) to know there are people like Tim Nichols and Dan Gunn at the User Research Group working hard to realize our wild and varied perceptions of great videogames.

Special thanks to Randy Pagulayan, Tim Nichols and Daniel Gunn for their help and enthusiasm.

Spanner has written articles for several publications, including Retro Gamer. He is a self-proclaimed horror junkie, with a deep appreciation for all things Romero.