To Hear Ourselves Review

On the face of it, it’s a simple system. Millions of gamers around the world turn to print and online sources for information on the newest games, and when they thumb through the pages of their favorite magazine or click around their favorite site, they see review scores, comparisons to other games and experiential analyses, all provided by game writers. But even in a relatively simple system like this one, there are wrinkles. And by wrinkles, I mean people.

From the audience to the developer, each title on the shelf represents the employment of hundreds, sometimes thousands of people, and the fate of entire companies often rests on whether or not you buy a game. Advertising plays its part in raising awareness of what’s out there, but even ads aren’t enough to sell games on their own. With hundreds of games on the shelves, the barrier to entry into your home isn’t just awareness, it’s also acceptance. And unless your game’s title starts with Halo or Madden, you need a good review (or preview) to prime that pump.

Reviews work like this: The developer makes a game. Once the game is finished (but before it ships), a PR representative (sometimes we call them “flaks”) sends a notice to various game writers and editors that a game is available for review. The PR flak sends out the game free of charge, then the reviewer plays it and writes several hundred words about it. The flak, having spread awareness about the game, considers his job done. Again, simple. Again, not really.

For starters, who is on the PR flak’s list depends on a great many factors, including but not limited to who he (or the developer he represents) happens to like.

The “No-Arthur” Policy
“I found out today that [the website] has a ‘no Arthur’ policy,” says Arthur, a veteran game reviewer working on the West Coast. His name isn’t really Arthur, and he’s speaking to us under conditions of anonymity.

Last year, Arthur received an assignment from a prominent editor for a prominent gaming publication to review a prominent game by a prominent developer for a prominent console. The review was scheduled to run alongside “exclusive features” offered to said editor by said developer. Merely because they were friends.

“This person actually makes no qualms over his relationship with the developer,” says Arthur. “He takes trips out to Japan and visits him while he’s there. He blogs all the time about being his best bud. [He] always seems to get exclusives out of them. It’s blatantly obvious, and he never makes any [bones] about it. He is [the developer’s] homeboy.”

Unfortunately, Arthur didn’t think the game was very good, so he gave it a low score and called it a day. Simple, but not.

“This is where it gets crazy,” Arthur says. “[The editor] sees this review and flips out. He calls [a] meeting. … He sits [the editorial staff] down at a table and says that this review is bullshit. … He demands that the score be raised.”

Although the rest of the team vouched for Arthur’s review score, the editor in question, concerned the low score would jeopardize his relationship with the game’s developer, refused to publish the review without an incremental increase in the score.

“I should have just told him to go fuck himself,” Arthur says. “But that’s the lesson I learned.”

Arthur agreed to raise his review score, but even with the requested changes, the finished review painted a less than flattering picture of what was to be the site’s featured game.

“[The editor] got on the phone, and he called the publisher and apologized,” Arthur says. “Then, he called the developer himself … and apologized to him.”

While Arthur’s experience isn’t unique, it is unusual. “Any player can write a review of a game,” writes Justin Hall, for Online Journalism Review, “but only sanctioned media outlets have access to games before they are available to the public. Brokering these agreements falls upon an untoward mix of editorial and promotions.” And these negotiations can often lead to long term relationships. Although most journalists and editors plead journalistic integrity in the face of payola or favoritism accusations, the most popular parties at E3, GDC and every other major game convention are the ones featuring an open bar and access to the developers.

Still, altering a review to avoid negative vibes is typically frowned upon, no matter whose game it is. The first question Arthur’s usually asked after recounting his story is, “Why didn’t this guy get fired?”

“I don’t have a solid answer,” he says. “It’s too baffling. I think the problem [with accountability] stems from the fact that hardly anyone even pays attention to who is writing [reviews]. So when a name actually pops up, they tend to stand out in a huge way.”

Which is another reason why, for this story, Arthur prefers to remain anonymous. Journalists prefer to report the news, not become it. Once you develop a stigma as a problem or unethical reporter, it’s hard to shake it. Especially once the developers know your name, too.

“Reviewers Want Every Game to be Zelda“
“A review serves one purpose and one purpose only,” says Warren Spector, the legendary designer of System Shock, Thief and Deus Ex, “to give readers data they need to make a buy/no buy decision. End of story. To do that, the reviewer has to have a consistent editorial stance. It doesn’t matter if you agree with a reviewer on a particular game or movie or book or record as long as they’re consistent enough that you can determine from reading the review whether you would like the game, movie, book or record yourself. Reviewers and readers have to develop an ongoing relationship of sorts. I don’t see that happening much in the world of game reviews.”

From the developer’s point of view, reviews are simple: They’re either good or bad. If the review is good, chances are sales will be good, too (exception: Psychonauts). If the review is bad, all is not quite lost, as sometimes even a bad review will move copies (see: most EA games), but it’s usually bad news. So, as far as a developer is concerned, there’s a lot riding on whether or not the person his PR flak sends a review copy to likes it. The frustrating part – for developers – is that no matter how much effort they put into a game, no matter how perfectly they polish it, there’s no guarantee they’ll receive a good review – or any review at all.

“Game reviewers want every game to be Zelda,” says Game Daily’s anonymous game journalism critic, Mr. Media Coverage. “That’s what one developer told me. He said that the reality of game development is that most developers make games for a very specific target audience, and the developers do their best to find and meet the needs of those specific gamers. It’s a frustration, then, when game reviewers complain that the game is too ‘kiddie’ or too ‘redneck’ or too targeted to one group. That, after all, was the entire purpose of the game.”

In spite of the fact you can do it in your underwear, reviewing games as a career doesn’t appeal to everyone, and the nature of the business, the requirement that reviewers be able to play (and often finish) hundreds of games per year, limits the available pool of talent considerably. This often means the folks reviewing most of the games are the people to whom most games are targeted, leaving niche games like Electroplankton and fl0w out in the cold.

“I’m starting to doubt whether games journalists should be the ones doing this job,” writes “Michael” at Tale of Tales, reviewing a review of fl0w, the game in which the player takes the role of a microorganism, instead of a beefy, armored space marine. “It’s a bit like having sports commentators criticizing a fine art exhibition.”

“In defense of the game reviewers,” writes Game Daily’s Mr. Media Coverage, “developers must understand that reviewers are writing for a target audience, as well. They have to target their content accordingly, and that occasionally means making fun of kids’ games.”

And, one assumes, experiential tech demos putting players in the role of a microorganism.

“I Don’t Know You”
“I don’t read game reviews,” writes Mike “Gabe” Krahulik (the artist half of artist and writer duo at Penny Arcade), in an article entitled “I Review a Review,” in which he utterly destroys a review of Enchanted Arms. “Regardless of how long [the reviewer] played Enchanted Arms it’s a worthless review. He didn’t like it for all the reasons I like it. At the end he attaches the number five like that’s supposed to tell me anything useful.”

Adding another wrinkle to the would-be simple system of game reviews is the issue of trust. In an age when nearly anyone can start a website and write about games, and most every type of gamer can find at least one popular message board (full of other people who are also playing games, and writing about them) to suit his needs, the need for formal reviews would seem to be diminishing. Or at least the need to trust the opinions of people you don’t know.

“I don’t know you at all,” Krahulik recently told The Escapist. “I don’t know what kinds of games you like. I don’t know how good you are at games. I don’t know what you want to get out of a game. I don’t know if you played the game because you wanted to or because it was laid on your desk. I don’t know if you rushed through it because the review was due on Monday. I don’t know if you only played the first two hours. I don’t know you.”

And yet millions of gamers still turn to print and online reviews to decide what to buy.

“[Reviews] are still, by far, the most popular articles in the magazine,” According to Jeff Green, Editor-in-Chief of Games for Windows Magazine, who calls reviews and previews the “meat and potatoes” of game journalism.

And Green and his readers aren’t alone. The developers are reading reviews, too.

“I try not to [read reviews],” says Warren Spector. “But, hey, you know, I’m human. I always end up giving in.”

Nokia producer Scott Foe reads them too. “Obsessively!” he says. “There’s an entire industry dedicated to judging your work – people who put food on the table by measuring your pecker for all the internet to read – you want to pay very close attention to the yard stick. That can literally be years of your life that those journalists are measuring. Take a second to ask yourself: What do the last three years of your life measure up to?”

For many developers (and journalists), three years can be a lifetime – or a career. But what happens when that three-year review comes in and the notes scribbled in the margin indicate your “pecker” needs a little work? As much as it hurts to see a project you’ve labored over for years criticized, when it’s fair, it’s fair. While it’s true people often have differing tastes, people also make mistakes. Convincing a developer to own up to them, however, can often be like forcing a horse to drink.

“The games press is not your enemy,” writes “Gazunta” at Angry-gamer.net. “Contrary to what you might have been told, the people reviewing your game actually want to see you succeed. Nobody wants to play a bad video game, much less people who spend all day playing bad video games. Most members of the games press are people who enjoy telling people about good games they got to play. Hopefully your game will be one of them.”

Foe tries to remain sanguine about criticism: “When faced with negative opinions,” he says, “it’s always important for you to focus on problems that the reviewer is highlighting, not solutions that that the reviewer is suggesting. (Henry Ford often joked that if he had listened to his customers, he would have built a faster horse.) There’s nothing more heartbreaking than when a reviewer of your work is both critical and correct – but there is also no one force more conducive to progress.”

Or, as Gazunta says: “Pro tip: Read the review. Take the criticism to heart. Do better next time.”

“The Opinion of a Stranger”
So what about the reviews, then? If the last three years of a reviewer’s life is represented by dozens of reviews of other people’s peckers, what does that say about the reviewer? And what can be done to improve the system?

As for the developers, they’d like to feel a reviewer is giving them a fair shake and not just punching a time card to cash a check – or to earn a trip to Japan.

“I get the sense that a lot of reviewers feel they have to appease publishers in order to continue to get the inside track on review copies, access to developers and so on,” says Warren Spector. “It’d be nice if that changed. … And I’d like to see reviewers express a more consistent … philosophical approach to games, so readers could find the [reviewers] they trust and stick with them a little more.”

Mike Krahulik agrees. “I’d try and create more personal relationships between reviewers and their audience,” he says. “That way, readers can determine if a given review has any value to them because they’ll know the reviewer. Otherwise it’s just the opinion of a stranger.”

As for the reviewers …

“Use lots of References,” writes “omni” from The Armchair Empire, in an advice column titled “How To be a Reviewer.” “Any reviewer worth his weight in games should be able to quote book titles, some Shakespeare. … Failing the ability to drop names and quote passages from obscure Russian plays or Monty Python, you can always mention every game you’ve ever played.”

While tongue-in-cheek, the advice does at least offer a solution to the problem of consistency …

“I would actually like more out of it,” says LevelUp’s N’Gai Croal, a bit more seriously. “I think as individuals we sort of have to say ‘Let’s [screw] around with the form more’ … let’s not get locked into debates. … Just rock out.'”

Which brings us back to you, the consumer.

“We can debate what art is, we can debate it forever,” said writer, director and erstwhile game designer Clive Barker at this year’s Hollywood and Games convention in Los Angeles. “But if the experience moves you, some way or another, even if it just moves your bowels, I think it’s worthy of some serious study. … Games aren’t about reviewers, they’re about players.”

And, in a strange way some would call “meta,” so are reviews. While the pundits, journalists, developers and flaks are debating the form, ethics and relative merit of reviews, the people at home are doing their own thing; which largely means taking what they want and leaving the rest.

“When I read reviews, I check the number score to see if it’s above a 7,” says Gamers With Jobs member Wordsmythe. “I then check [the] list of the good, bad and ultimate take on the game. If I’m still intrigued or confused, and if I have free time, I’ll skim the first few and last paragraphs.”

Russ Pitts is an Associate Editor for The Escapist. His blog can be found at www.falsegravity.com.