Exterminate!

Like it or not, bugs are a fact of life. As the saying goes, even the cleanest house has bugs. Just ask the Waldorf-Astoria hotel. Software is no different. Even the best-tended, most carefully-coded pieces of software have bugs, large or small, that can manifest in a multitude of ways. Some hardly noticeable. Others, catastrophically.

A hotel like the Waldorf-Astoria collects bugs much in the same way a software program does, by being exposed to users and environmental conditions unable to have been foreseen by those who constructed it or are responsible for its upkeep. In a hotel, this may mean a random guest checking in with baggage infested with bedbugs from another hotel. Likewise, bugs may manifest when software is used by someone with a piece of hardware the software’s creators didn’t know about. Or when the user tries something the software coders never imagined they’d try. Or, as in the case of the infamous AT&T network outage of 1990, when the software performs an operation impossible to test before release.

In December of 1989, AT&T released a software upgrade for the machines that routed long-distance telephone traffic. The upgrade was supposed to streamline the way these 114 trunk switches managed traffic, but instead introduced a bug that crashed the whole system, bringing the AT&T long distance network to its knees.

The way the system was supposed to work, if one switch went down, it sent a message to its neighboring switches, telling them it was out of service, before attempting to reboot itself. When it came back online, it then would send out a message telling its neighbors it was back in business. This was supposed to allow the system to rapidly adjust to fluctuations in service and remain stable, even when part of the system was offline.

The bug in the code, however, caused a switch’s neighboring switches to crash upon receipt of the “I’m back online” message, forcing those neighboring switches to crash, reboot, then send their own “I’m back online” message, thus causing their neighboring switches to crash. The result was a chain reaction of repeating system failures bringing the whole system down for nine hours. Engineers finally had to manually reduce the load on the switches, reboot the entire system and reload from an older version of the software.

The moral of this story: If it can happen to AT&T, it can happen to you. Games are nothing but highly complex, cutting-edge pieces of software designed to push the envelope of whatever hardware is running them. Most pieces of game software are intricate, produced by multiple coders working with proprietary tools on tight budgets and extremely strict deadlines – all conditions that serve as the perfect breeding ground for an infestation of bugs.

This week, we’re devoting our issue of the magazine to bugs, with Issue 289, “Exterminate!” Brendan Main explores “The Minus World” at the edges of a game’s creation, William Bloodworth examines those rare “beneficial” bugs, and for my contribution, I take the reader on a tour of the bugs to be found in the first 100 hours of Bethesda and Obsidian’s Fallout: New Vegas. Enjoy!

/Fingergun