Killer Robots and Collateral Damage

The U.S. military currently has 7,500 drones. They do everything from search and rescue, to scrambling enemy communications, to icing militants in remote areas of Pakistan. Like all military technologies, they’re both problematic and full of possibility, simultaneously praised for keeping troops safe and characterized as “videogame warfare,” where joystick-wielding pilots kill human-shaped blips on a screen. Perhaps with these comparisons becoming common, it was inevitable that games would cast a critical eye on the topic of remote warfare.

While drones appeared in a number of games this year, three in particular – Spec Ops: The Line, Call of Duty: Black Ops II, and Unmanned – contain cautionary tales about the problems of remote warfare. Interestingly, these games broke in different directions with their criticism: Spec Ops warns about the problematic nature of target selection from a distance, Black Ops II concerns itself with the looming issue of autonomy, and Unmanned looks at the emotional cost to drone crews. In all cases, the interactive nature of virtual experience gives the player a unique perspective on the topic.

“Are Those … Civilians?” (Includes spoilers for Spec Ops: The Line)

In the most memorable scene of Spec Ops: The Line, Captain Walker and his men come upon a company of rogue U.S. Army soldiers garrisoning the gates of downtown Dubai. Heavily outnumbered and outgunned, Walker orders his team to man a white phosphorous mortar and clear out the enemy, with Walker himself (and by extension, the player) calling fire from a UAV. Afterward, the team walks through the devastation they’ve caused – charred corpses, burning vehicles, soldiers with missing legs begging to be shot – and finds out that they’ve also killed a large number of civilians the troops had in a holding area. The defining image of the game is a woman scorched down to the muscle, cradling her dead daughter with a hand over the child’s eyes.

While the scene itself is a direct criticism of military shooters in general, (like gamers, Walker is perpetrating violence against anonymous avatars through a screen – a sort of game-within-a-game) it’s also a commentary on the United States’ recent reliance on UAVs. By making the player responsible for calling the targets, the designers directly confront the player with the idea that enemies and civilians may look so similar from a drone’s perspective that they’re rendered indistinguishable – that the camera dehumanizes everyone down to white, vaguely-human blobs that a user is more likely to kill without hesitation.

It’s a problem that’s plagued the U.S. UAV program, especially in countries like Yemen, Somalia, and Pakistan. Directed by the CIA rather than the Pentagon, the purpose of the U.S. drone campaign in these three countries is to target and kill the leadership of Al Qaeda and its regional affiliate groups, thereby disrupting and dismantling terrorist command structures. The program itself has been highly prolific: To date there have been 354 drone strikes in Pakistan, between 42-52 in Yemen, and 10-23 in Somalia. Targeted killings have been especially successful in Pakistan, where drones have battered Al Qaeda networks in North Waziristan to the point where they have difficulty recruiting and training new fighters. Just this month, a Predator UAV killed Al Qaeda senior leader Seikh Khalid Bin Abdul Rehman Al-Hussainan as he ate breakfast, denying the organization of yet another probable successor to al-Zawahiri (it’s happened many times before – there’s a joke in the Middle East that being named second-in-command of Al Qaeda is the fastest way to die). This kind of success is why the Obama administration has largely turned to UAVs as its primary terror-fighting weapons – they’re an expedient solution to a complex political problem. Drones allow Obama to kill terrorists hiding in foreign countries while avoiding the following irksome political issues: declaring war, risking American lives, terrorism trials, and prisoner incarceration. As a result, drones have increasingly become Obama’s favorite weapons in the war on terror – and according to some reports he compiles the “kill list” personally.

But while militarily and politically useful, evidence indicates that the drone programs take a heavy toll on civilians. While hard numbers are difficult to come by, an ongoing report from the Bureau of Investigative Journalism claims that of the 2,597-3,398 people killed by U.S. drones in Pakistan, between 473-889 of those have been collateral damage. (A different accounting of civilians killed in Pakistan puts the number at 17 percent of total drone casualties during the Obama administration, down from around 40 percent during the Bush years. It also suggests that fewer civilians are killed each year as strikes become more surgical.) Partially, collateral damage is a consequence of counterinsurgency warfare – which by definition means fighting an enemy that lives amongst the local population – but it is likely exacerbated by the drones themselves. While most drone attacks are sold to the public as “personality strikes” that target prominent terrorists, the CIA also conducts “signature strikes,” attacks based on surveillance of groups of unknown persons who exhibit behavior patterns that could be interpreted as indicative of terrorist activity. In a remote region like Waziristan where the United States does not have a great deal of human intelligence assets, this surveillance is largely conducted via drone camera feeds – heavily increasing the chance of target misidentification.

In fact, the difficulty of telling militants from civilians has made waves at the White House. According to a New York Times story based on several White House leaks, President Obama has embraced a controversial method of differentiating which targets are legitimate: Namely, all military-age males killed in U.S. strikes are labeled combatants, unless posthumously exonerated by new intelligence. Advocates of the policy say it’s based on sound logic – that Al Qaeda is an insular organization and innocent people don’t hang out around them – but human rights advocates and dissenting White House staffers contend that it’s a form of guilt by association and deprives people of their right to life without due process. The situation gets even murkier when you take into account other drone tactics. Not only do U.S. drones sometimes target funerals (in hopes of killing associates who show up to bury dead militants) but they also engage in “double tap” attacks, where the drone circles around to attack a site again – targeting anyone who comes to the strike zone offering assistance to casualties of the first strike. Not only does this tactic increase the possibility of killing civilians, but by targeting people rendering medical aid, it may actually fall under the definition of a war crime. In fact, there it’s possible that drone strikes are actually counterproductive, since they’ve become a major recruiting tool for terrorist groups.

While there are many ways to talk about the problems inherent in selecting a target via drone, Spec Ops illustrated the concept in a way that only games could – by making the player part of the decision themselves.

“I’m Don’t Worry About the Guy That Wants to Hijack a Plane, I Worry About a Guy Who Wants to Hijack All the Planes”

Despite its absurd alarmism, the quote above – said by Oliver North in one of Black Ops II‘s widely-criticized pseudo-documentary ads – nails the game’s primary theme: What happens if we lose control and our drones turn against us? Essentially, Raul Menendez’s plot boils down to turning America’s high-tech weaponry against it as punishment for the misery the U.S. caused through Cold War proxy conflicts. (At least that’s the most sense I can tease out of it. The plot of Black Ops II is convoluted at best, and at its worst is just a litany of contemporary shout-outs.) However, Black Ops II also deals with the issue of drone autonomy, a question that may become one of the defining ethical debates of the next twenty years.

Black Ops II depicts drone autonomy as neutral but problematic. Drones under the player’s control serve as a potent force multiplier for the JSOC, allowing Section and his team to take on much larger forces. However, the game also explores the horrifying potential that drones might be badly programmed or fall into enemy hands. During one mission, a hovering drone sweeps the flooded streets of Lahore as a pre-programmed death squad, killing anyone caught in its searchlights. Later on, Section and his team try to stop a terrorist attack on a floating resort in the Cayman Islands, only to have autonomous security drones open fire on them, since the robots perceive anyone with weapons as an enemy. Overall, the game presents autonomy as a useful technology but warns that its imperfections could render it extremely dangerous.

Drones currently have a limited amount of autonomy, mostly in regard to stabilizing the aircraft and plotting flight paths. However, the technology is improving fast, and with each generation drones take over more tasks from their human handlers. This year the Navy began testing a stealth drone called the X-47B that can take off and land on a carrier deck with the click of a mouse and perform pre-programmed missions without human guidance (though its systems will be monitored by a human). These are the type of drones Black Ops II portrays – fully autonomous or near-autonomous combat systems that can chart flight paths and engage targets independently of an operator – and it’s this degree of autonomy that worries experts both inside and outside the military.

One ethical quandary is one we’ve already discussed: how autonomous drones will differentiate between combatants and non-combatants. Will targets be selected and neutralized through internal systems, or by the human monitor? Suppose a drone identifies a high-value target but there are civilians in the strike zone – will programmers need to write an algorithm that allows a drone to calculate how much collateral damage is too much? And if civilians are killed by an autonomous drone, who can be held accountable for that action under international human rights law? Subject to the specific circumstances of an incident, you could make equally strong arguments that responsibility lies with the drone’s commander, mission programmer, or even manufacturer. These questions seem like the outline for an Asimov novel, but the issue is real enough to warrant a 50 page report from Human Rights Watch titled Losing Humanity: The Case Against Killer Robots, where the organization urges an international ban on autonomous drones due to risks to civilians and the accountability gap they create. While laudable, an international ban would prove impractical for a number of reasons – mostly because autonomy is a pretty elusive concept to define technically, and would be even more so in drafting an international agreement (the UN has been trying to create an internationally-accepted definition of “terrorism” since 1937).

However, the U.S. military shares some of the same concerns as human rights groups, and is taking its own form of action. Last month, Deputy Defense Secretary Ashton Carter signed a series of instructions that would prevent the military from purchasing autonomous or semi-autonomous drones “that could lead to unintended engagements,” meaning drones that target and attack of their own accord. Basically, the Pentagon believes that the determination to engage a target should always come from a human and be part of the military’s traditional command structure, rather than an automated process. The debate doesn’t end there though – the instructions specifically allow autonomous systems relating to surveillance, cyber weapons like Stuxnet and Flame, deploying mines, and unguided and operator-guided munitions.

But what are the odds of these newly-autonomous drones being turned against their operators? Well, for now the chances are pretty slim. There have been some data breaches of U.S. drones, most notably in late 2008, when U.S. forces in Iraq arrested a Shiite militant who had a laptop full of intercepted camera feeds from a Predator. The next year, American troops found more laptops with hours of drone footage, and after an investigation, determined that Iran is teaching militants how to intercept the unencrypted feeds with a $26 piece of hardware. Iran, in fact, has met with some success countering the U.S. drones that cross its border to spy on its uranium enrichment facilities. In late 2011, Iran captured an RQ-170 Sentinel drone that was likely operated by the CIA. While the circumstances surrounding the capture are still unclear, Iran claimed that they jammed the Sentinel’s GPS signal and then spoofed it, tricking the Sentinel into thinking it was back home and landing at an Iranian air base. U.S. experts, on the other hand, find that claim dubious, since there’s no indication Iran possesses that level of capability, and say the drone probably crashed due to pilot error. However, those experts also admitted that the exploit Iran described is a real problem and the scheme would be possible, and stress that fully hijacking and controlling a drone via computer is still in the realm of science fiction.

Call of Duty: Black Ops II isn’t focused on giving us a thoughtful experience regarding emergent technology, it mostly just wants to give us cool things to shoot at and cool weapons to shoot at them with. However, despite its bombastic storyline, it does raise interesting points concerning drone autonomy and the dangers we face by letting an automated system determine who lives and who dies.

“Isn’t That Like Playing With a Cheat Code?”

Finally, the most incendiary criticism came from Unmanned, a flash game by Molleindustria where the player takes on the role of a drone sensor operator for the U.S. Air Force. Unlike Spec Ops and Black Ops II, Unmanned takes place in the present day, and focuses on the daily stresses drone crews face – long, dull hours staring at a screen, life-or-death decisions, bizarrely sanitized violence, and returning to a “normal” family life at the end of each shift. The main character of Unmanned awakes each morning from a nightmare of being chased by Afghan villagers. He shaves and drives to work, flirts with his co-worker, stalks people thousands of miles away, calls his wife while smoking a cigarette, kills a man planting an IED (or not), then chats with his jingoistic son while playing Call of Duty parodies. It’s the last scene that throws the character’s job into stark relief – videogames are the only time the enemy shoots back at him, a contrast to his day job, where he’s never at risk of enemy fire. His son likens it to playing with cheat codes, a clever verbal flourish that asks how war changes when some combatants don’t have skin in the game.

Despite these larger philosophical questions, Unmanned‘s focus revolves around the drone crews and their feelings. The greatest sense you get from playing the game is fatigue. It’s an easy game, devoid of challenge and excitement. Shaving and driving to work are more difficult than launching missiles, which is entirely the point. Ultimately, Unmanned argues that dealing with the stresses of combat aren’t what affect drone pilots, it’s the dichotomy of fighting a war on Friday and fixing your son’s bike on Saturday.

In reality, there’s evidence that drone crews do face a high level of pressure from this new mode of warfare. A recent study by the Air Force School of Aerospace Medicine found that 46 percent of Reaper and Predator pilots and 48 percent of Global Hawk sensor operators reported “high operational stress,” mostly from long hours and shift changes. Only 4 percent of the pilots surveyed were considered at high risk of post-traumatic stress disorder, though one of the report’s co-authors, Colonel Kent McDonald, stressed that the virtual nature of drone warfare doesn’t shield pilots from the emotional cost of combat. “Killing in and of itself is difficult for any warrior to go through,” he said in an interview with The Huffington Post.

Compounded by poor ergonomics and user interfaces, the environment in most drone control trailers is both uncomfortable and unintuitive. Unlike Unmanned suggests, flying a drone isn’t easy – the flight instruments are confusing and badly laid out, operators monitor multiple data feeds simultaneously, and getting clearance to fire means juggling calls between a military JAG who okays it under the Rules of Engagement and the Combat Air Command Center in Qatar that makes the final call. Then, after it’s done, you drive home without the military’s traditional support network. “When I was a pilot,” remembers Dr. Missy Cummings, a former Navy pilot and professor at MIT, “you came back from a mission, you would come back to the carrier to be with people who were doing the same thing you were doing. You were all together in it. On your own, it’s harder to keep it in perspective.”

However, contrary to the bleak message of Unmanned, the Air Force study found that drone crews were highly satisfied with one aspect of their job: providing air support for ground troops. These more traditional air strikes – relieving troops pinned by enemy fire or taking out mortar positions bombarding a base – have led to a weird camaraderie between combat infantry and their virtual protectors in Nevada. Drone crews sometimes receive emails from infantrymen after a strike, thanking them for their help. “They love that,” Colonel McDonald told the New York Times. “They feel like they’re protecting our people. They build this virtual relationship with the guys on the ground.”

Regardless of their faults and the ethical dilemmas they present, the overwhelming military usefulness of drones means they’re not going away. Though some reports claim that the military is starting to scale back its commitment to remote warfare in the short term, it’s likely that unmanned vehicles – maybe autonomous, maybe not – will be fighting our battles for a long while to come. How we will handle that remains to be seen, but with the interactivity becoming an increasingly powerful rhetorical tool, it’s likely that this is not the last time we’ll see games that critique this new facet of warfare.

Robert Rath is a freelance writer, novelist, and researcher based in Austin, Texas. You can follow his exploits at RobWritesPulp.com or on Twitter at @RobWritesPulp.