Is the World Ready for Deep Network AI Opponents?

This week someone sent me a really interesting question on AI in games:

Hi Shamus,

I’ve been reading and hearing a lot about AI. Where deep networks do the hard heavy lifting of processing data then output relatively lightweight AI data… such as Google’s Tensor Flow (now open source). There are numerous videos on YouTube of AI learning to play video games (There was a fantastic one on Ted Talks but I can’t find it now).

Do you think this is a viable way to create AI foes in games?

Rick Hambrook

This is the part where I usually put on my programmer hat and say something like, “Based on my years of programming, I think that…” Except, AI isn’t in my field of expertise, and every single time I write about AI programming I get nitpicky emails from dedicated AI programmers telling me I have no idea what I’m talking about. The last time around, I called out the marketing shills who claimed that the next-gen consoles would give us powerful new AI opponents, saying that AI doesn’t need more CPU, it just needs people to be willing to spend the money to make it happen. And the response I got from AI programmers was basically, “You don’t know what you’re talking about.”

It would be arrogant of me to dismiss these folks as a bunch of over-sensitive whiners. The world of programming gets bigger every single day, and it’s hard enough for specialists to keep up with the changes in their own field, much less keeping track of what everyone else is doing. So I’m going to assume these other programmers know their stuff and indeed are privy to advances that I don’t know about.

But ignoring my alleged programming expertise and speaking strictly as a consumer and a player: I am not seeing those advances on my end. AI programmers may indeed be inventing brilliant new techniques to make videogame opponents smarter, but for the average player it’s just not visible. Marketing always promises smarter foes when a new console generation comes around, and yet the dudes we’re shooting today feel pretty similar to the dudes we were shooting a decade ago. You could even argue we’ve gone backwards a bit. F.E.A.R. came out in 2005, and it’s still the high-water mark for me.

To be fair, games have gotten more complex. There are a lot more open world games and a lot of them have complex traversal mechanics (the recent Saints Row and Just Cause entries come to mind) that might present situations that are hard for an AI to handle reasonably. For example: If someone ran up to my car and grapple-hooked it to a cow and then put rocket boosters on my car, I’m not sure I could come up with an intelligent response, so I can’t fault the denizens of Just Cause 3. So maybe AI programmers are fighting hard just to maintain the same baseline level of cannon-fodder dipshit foes in a world of ever-escalating complexity.

But coming back to Rick’s question: Could we use deep networks for videogame?

This ties into Moravec’s paradox, which is longstanding problem in AI development that states roughly: Things that are easy for people are hard for computers, and vice-versa. If I give a person the weight and starting velocity of a projectile, they will likely have a very hard time working out the parabola. A majority of people won’t even know how to calculate the answer, and the remainder will need some time and scratch paper to work it out. A computer? Done in less than a millisecond.

On the other hand, try this task: Have the subject examine several photographs and identify if the photographed object is a cup or a hat. That’s trivial, even for a small child, and was basically impossible for computers until very recently. We didn’t solve this problem by hiring a programmer to come up with a “hat detection algorithm”. Nobody turned the concept of a hat into 3D models and source code for the computer to consider. Instead, they just gave the program thousands of source photographs of hats and had it look for similarities. That takes a lot of processing, which is why it didn’t happen until we could build huge farms of data-crunching computers.

I can’t imagine how this could benefit the world of action games, but it could be a really interesting research project for someone working on large-scale strategy games like Civilization or Total War. Writing AI for those games is brutally difficult. Like I said a few years ago, combat AI starts out perfect and the programmer has to work to make it fallible, while strategy AI starts out stupid and the programmer has to work really hard to make it even a vague challenge for a human. Playing Civilization on a human level is almost a “is this a hat or a cup?” type of problem, and I’m amazed that the AI in these games can give the player an interesting challenge at all. (Although the higher difficulty levels often need to cheat to keep up with the player, either through granting itself extra resources or by looking at information that wouldn’t be visible to a human.)

Using deep networks would represent an entirely different approach. Instead of the developer trying to turn a few strategies into code, they would instead feed the program the history of millions of turns across thousands of games. Again, this is not my area of expertise, but as far as I understand things this would be something you couldn’t do on the user’s machine. You would need some central repository and a beefy server to crunch the data.

While playing, the AI would look at the state of the game, compare the result to the historical record, and pick behaviors that have performed well in those conditions in the past. Perhaps when a game ended, the move history could be sent to a server. Over time, the AI would get smarter as the number of historical games in its library grew.

I’m reasonably confident this is feasible with current technology. The trick is figuring out how to make it work in a game. Assuming the database is too large to distribute to users (although games are getting pretty damn big these days, so I don’t know) then you’ll need to have a central server. Your computer AI opponent would send a message to the server saying, “The game looks like this right now.” and the server would respond with (say) a list of priorities for what the AI should be doing or building. The last time someone took the CPU load off the user’s machine on put it on the cloud, things didn’t go very well. Remember the SimCity fiasco?

On one hand, the AI would only need to check in once every few turns, so the server load would only be a fraction of what SimCity needed. On the other hand: If you get this wrong and the server can’t keep up, you end up with huge controversy, review-bombing, brand damage, low sales, and bad press.

It wouldn’t be as easy as I make it sound. You’d need some way to allow for the very important fact that most players don’t want to face the toughest possible AI. This means your incoming flow of data isn’t from people playing optimally, and your output to the AI needs to be (somehow) less than optimal. This will corrupt and muddle the flow of data in all kinds of subtle ways. You’ll end up teaching the AI how to be a halfhearted, below-average player. Which, fine. That might be what people want. But if the AI isn’t trying to be exceptional then why not just save yourself the effort and stick with the AI systems we’re using now?

So here is the answer to the question: To use deep networks for game AI you need to…

1. Spend money to research and develop an entirely new form of game AI…
2. that depends on a large server farm…
3. to run the AI for a fundamentally niche (by AAA standards) videogame…
4. for an audience that might not want, need, or even notice the feature…
5. which some people will HATE no matter how well it works because they oppose “always online” for single-player games on a fundamental level…
6. and which will incur bad press financial loss if anything goes even slightly wrong.

I think the technology is either ready, or pretty close. But making a business case for it is going to be tough.

(Have a question for the column? Ask me!)

Shamus Young is a programmer, critic, comic, and survivor of the dot-com bubble.