Mario Reads Minds

San Francisco startup Emotiv Systems emerged from the shadows at GDC 2007 with a rather startling announcement: They’re working on a way to read your mind, and they’re close to release. Big, brassy announcements with nothing behind them are the bread and butter of the gaming industry, the better to be shoved back in a company’s face when things don’t work out, but Emotiv seems to be a cut above the usual design-doc-and-a-prayer windbags.

For one thing, the pedigree of the Emotiv team is impressive, made up of bright stars of business and award-winning scientists, engineers and executives. The company’s founders include: Allan Snyder, a noted neuroscientist and Marconi Prize winner, as well as a pioneer in the field of fiber optics; Tan Le, a rising young technology star and entrepreneur from Australia; Nam Do, a partner of Le’s and a gifted student and manager; Neil Weste, one of the leaders in chip design; and Steve Sapiro, a heavy hitter from Intel with a passel of companies under his belt. Sitting on the Board of Directors are two more notable names: John Murray, a heavy hitter in Australia’s financial scene, and Ed Fries, formerly one of the head honchos in Microsoft’s game division and the reason you’ve heard of the Xbox.

Finally, Randy Breen is a game industry veteran with a pedigree to die for – 14 years as a producer and executive producer at EA and four and a half years as a Vice President and Head of Development at LucasArts – who now serves as the Chief Product Officer for Emotiv Systems. Breen’s job focuses on “extending the research into new areas and figuring out how to productize that research, both from a hardware and software standpoint, and to evangelize it, so that developers understand how to use it. Basically, [to] get SDKs in their hands early.” While they’ve been primarily working with developers, they plan to get a device out to consumers “sometime next year.”

According to Breen, the company is three years old. “Two of the founders [Le and Do] were acquainted with a fellow named Allan Snyder … and a fellow named Neil Weste,” he says. “The four of them got together and talked about the potential for creating a device that could tap into unconscious thoughts and use those as a new form of interface for computers, and that lead to about a year and a half of research.” The fledgling company recruited Breen for his game industry experience, as the founders felt gaming was a logical fit for the research. Their work produced a headset that recognizes brainwaves and patterns and the Emotiv Development Kit, which lets developers integrate the information from the headset into the games they make, detecting thoughts, emotions and other kinds of brain activity.

“We’ve announced the release of a software development kit,” he says. “The kit is really a subset of what we think will come to market for consumer applications. It gives a sense of direction, and the idea is for us to get that equipment and our software into developer hands early, so that they can start to build applications that utilize these things.” Emotiv currently offers three software suites, “the Affectiv Suite, which is emotional detection, the Expressiv Suite, [which] is facial expression, and the Cognitiv Suite, [which] is conscious decision.” Using the Expressiv Suite as his example, he says, “The idea is to be able to express yourself naturally rather than using emotes.” So rather than typing /smile or /wink, “you might just smile and wink, and be able to translate your physical actions into actions that your character online, or your in-game character can [use to] communicate with another, non-player character.” The Affectiv Suite detects emotions. “We released excitement as the first example of that. The idea is that we can detect a range of emotional behavior, from an excited state to a calm state, so that those can be used to trigger events in a game, like control of an audio score, or [translating] your actions onto your character, so that not only does your character respond through your facial expressions, but it might respond through its animation, using your feelings.

“Calming could be used to provide health rather than picking up power-ups or waiting on timers, potentially,” Breen continues. “You could play out the fantasy by having to get out of the line of fire and relax somehow.” On the other hand, feelings of excitement “could be used on the other end of the spectrum to enable the character, so if the fantasy of the character was to become empowered by becoming excited, then, again, you’re feeling the character’s role rather than simply forming mechanical actions in the game. And we also think that the emotional detections have the potential to be used to enable dynamic difficulty in a game. So, for instance, being able to make adjustments to suit the player’s ability or their feelings, rather than predefined specifications for what difficulty should be.”

Excitement may be where they’ve started, but by the time they bring their offerings to market, they hope to offer the ability for games to “get a sense of what the player needs. You need to adjust the game to suit the players. And we think that that’s really important, because the range of player ability right now is so broad that the games have difficulty really tailoring [themselves] to suit that range of ability.” Emotiv’s suites offer game developers the ability get “a sense of … how the players are responding to the content. We think that’s a really powerful tool to provide the best possible experience.” He makes a comparison with film editing, where “a director has a very fine control of tempo by carefully editing the scene. They’ll basically get the audience to an apex and before they deliver new material, they want to make sure that the scene provides time for the observer to relax and settle before they provide the next big bang.” Games don’t currently have this luxury, he says, and Emotiv “provides a method for game developers to sense whether the player is at the emotional level they want them to be [at] and then [they are] able to adjust the content accordingly.”

Their third offering is the Cognitiv Suite, which allows the game to detect conscious decisions from the player’s brainwaves and interpret them into in-game actions. “You can visualize an action and see that action translated into movement on screen,” Breen says. They’ve already got some basic movements in, such as lifting, pushing, pulling, dropping and rotating an object, “and those actions can be translated to anything onscreen, really, that you might use them for.”

The science may be complex, but the interaction is simple: “You think about pushing the object, and the device allows detection for that event, and then you’re able to perform that action [in game].” Currently, they’re up to being able to detect three distinct actions simultaneously, which he says is “just the beginning. It’s not clear where the limits are, but we intend to continue and expand both the number of things that you can do, [both] the number of things that you can do simultaneously, as well as the list of things you can do discretely.”

Developers hoping to work with the system and utilize Emotiv’s development kits “will find it extremely easy to work with,” he says. “The detections themselves have been done, so the information that’s passed through the API is fairly standard and easily integrated into menuing and functions that they would expect. It’s not going to look very different than other types of interfaces, other than the fact that it’s detecting these discrete things that they didn’t otherwise have access to. From a functional standpoint, it’d be very simple.” No mind reading required.

As for players who will – hopefully – be encountering Emotiv’s hardware and software packages, Breen is aware of the “gimmick” status of the similarly innovative Wii controller in the minds of some of his audience. However, he doesn’t think they’ll run into the same resistance, he says. “The nature of what we’re doing is providing a method … for players to become more immersed in the material that they’re already interested in. When you think about the trends, there’s a long-term trend toward realism and toward more complete experience in the game world. When you think about your expressions and your feelings being part of the scene, this is really providing an avenue for that relationship.”

Instead of typing slash commands or emoticons, “now you’re just smiling. And when you’re excited, your character demonstrates excitement by just changing his body language, or the music changes. It’s just a reinforcement that that’s how you feel, increasing tension the way films do. Or, in the case of the cognitive actions where you’re imagining pushing an object with your mind, you’re really mimicking the things characters are doing in the game. Whether that’s magic or telekinesis, it’s fantasy fulfillment.” He thinks taking out the middle man and taking out, say, button presses, “which are not very good representations of the actions they represent,” would make for a “much more fulfilling experience.”

However, Emotiv isn’t seeking to change things completely, he says. “I imagine that the device is a supplement to the joystick. It’s not trying to replace it. The conscious actions, the nature of them is not as immediate as the joystick control, but it provides a level of satisfaction that’s different. … It’s a different type of experience.” And one that may completely change the way we play.

[em]Shannon Drake is a Contributing Editor for The Escapist and changed his name when he became a citizen. It used to be Merkw