Dealers of Lightning

by Michael Hiltzik

  For the first hour or so no one even suspected that the crashes might be part of a more general crisis. But it soon became obvious that the Altos had been infected by an entirely new quirk. They booted up fine, but step away from your machine for even a few minutes—and it crashed again. Reboot again, fine. Leave it idle . . . and death. Soon the first and second floors of 3333 Coyote Hill Road were fairly echoing with voices raised in apprehension and protest.

  Hey, what's the matter with my Alto? . . . Something go wrong with yours, too? Mine was dead when I came in!. . . Mine too!. . . And mine!. . . What's going on? There must have been a hundred machines crashed when we came in this morning!

  At length someone in CSL put two and two together. "You know, Shoch and Boggs were here all night."

  "Working on what?"

  "Something to do with the Ethernet. . . ."

  And so the summons went out. John Shoch, who had indeed been working all night on a network diagnostic program with Dave Boggs, had scarcely dropped off to sleep when he was jarred awake by the fire call from PARC. Machines were dead all over the building. Did he happen to know anything about it?

  Uh-oh, he thought to himself. Something must have gone wrong with the Worm.

  By the late 1970s much of the work in PARC's two computing labs entailed elaborating on the inventions produced during its first few glorious years. Inventing the Alto and Ethernet was like seeding a field. Now it was time to harvest the crops—in other words, to look more closely into how these systems worked in the real world and in concert.

  John Shoch decided to tug on one of these threads for his Stanford doc­toral thesis. His plan was to study how a network behaved under various message loads and traffic patterns. But it would not be enough to watch it operate under normal conditions. The really interesting thing would be to see what happened if you really pounded on the system, loading it to the absolute limit to see if it blew a gasket.

  He knew he had a major advantage over other students of network the­ory. He had daily access to the Ethernet, then the world's largest and busiest computer network. The only question was whether Xerox would let him publish what he learned.

  This was not an idle consideration. Xerox was acutely aware that its wealth flowed from secrets and property rights. Behind its domination of the copier market—which was waning but still strong—were a handful of ferociously defended patents, as well as other inventions so critical they had never been exposed to the public scrutiny of a patent application. Instead they were locked in a vault as secure as the one that safeguarded the recipe for Coca-Cola. By 1973 the company had assigned a corporate patent lawyer to PARC full-time and decreed that even technical papers were to be vetted by counsel and cleansed of any inadvertent proprietary disclosures before they could be submitted to professional journals.

  On rare occasions PARC issued technical papers on its own. These were known as "blue books" from the color of the horizontal stripes across the covers. But for the most part the legal department controlled the publication of research with an iron hand. So much creative talent went into the building on Coyote Hill Road and so little information came out that PARC, recalled one former manager, was sometimes called "the black hole of computer science."

  PARC researchers participating in conferences and other public events had to carefully comply with the company's paranoid rules. They did not always avoid embarrassment. One of the more delicate situations involved the Ethernet team of Shoch, Bob Metcalfe, and several other engineers. As an ARPANET host location PARC was honor-bound to help address any technical problems cropping up during construction of the nationwide system. Unfortunately, Xerox's lawyers had given the Ethernet engineers strict orders to keep to themselves everything they knew.

  This quandary first surfaced in 1973, when Metcalfe attended an ARPANET conference at Stanford. One issue under discussion arose from the proliferation of independent "local area networks," or LANs, similar to the Ethernet at PARC. These networks' technical standards, or protocols, were often incompatible with each other and with the ARPANET itself. The question was how to coordinate them so data could smoothly pass from one to another via the ARPANET backbone— a critical step toward ARPA's goal of expanding its network into a larger and more comprehensive system linking computers everywhere—trans­forming it, in other words, into an "internet."

  For the ARPANET's technical honchos this question was still largely hypothetical. So far there were only a few operating LANs sophisticated enough to tap into the main network. For PARC, however, it was already a pressing issue. At Stanford Metcalfe held his tongue, but he returned to his office convinced he could not wait for ARPA to solve the problem. Xerox was already planning to use the Ethernet to link corporate loca­tions all over the country—indeed, the globe. Data packets would have to traverse outside systems on their way from one local Xerox loop to another, like trucks hopping on the interstate en route from California factories to New York retail shops.

  "We have a more immediate problem than they do," Metcalfe con­fided to Shoch, "because we have more networks than they do. We're going to have to build an internet protocol ourselves."

  Working with Boggs and Ed Taft, a mathematician from Harvard, Met­calfe and Shoch came up with a solution they called the PARC Universal Packet, or "Pup." The basic idea was to enclose a data packet in a sort of electronic envelope consistent with the protocol of the transporting net­work—radier like a child's birthday present wrapped and tagged with a Federal Express airbill for transport by its planes and trucks. As the packet moved from network to network, it could be further wrapped in any number of network-specific capsules. By the time it reached its des­tination the various envelopes would have been stripped away, leaving the original message to be read.

  In the year they spent developing and testing Pup over dozens of vari­ant networks, the PARC engineers moved well ahead of the ARPANET team in their technical expertise. This was unsurprising. Their sandbox, comprising nearly 500 host computers scattered around the country, was already much bigger and busier than the ARPANET.* Moreover, PARC was immune to the ARPANET's lumbering bureaucracy.

  "We had the great advantage of getting paid to do this full-time for a liv­ing," Shoch recalled. "The ARPANET guys were working under govern­ment funding and university contracts. They had contract administrators and students working for them and all that slow, lugubrious behavior to contend with. We had a lot of resources and a lot of machines and we didn't have much else going on. We were evolving ahead."

  *It would be several years before overall activity on the ARPANET significantly sur­passed what PARC generated within its own building. As late as 1979 the average daily traffic on the PARC Ethernet, which linked 120 Altos and Dorados, came to fully half what was carried nationwide on the entire ARPANET.

  But when ARPA invited them to a technical meeting to work out the very same issues they had already solved, they faced a painful dilemma. Xerox's lawyers, still deliberating whether to patent Ethernet or market it as a proprietary product, placed the PARC team on a short leash. "We were told to participate in the meetings," Shoch said. "But we were ordered not to describe what we were doing."

  Nevertheless, they felt a powerful urge to impart their wisdom to their friends at ARPA. Thanks to the legal beagles' strictures, they were reduced to getting their points across by a weird pantomime of asking inscrutable but cunningly pointed questions. "Somebody would be talk­ing about the design for some element and we'd drop all these hints," Shoch recalled. "We'd say, 'You know, that's interesting, but what hap­pens if this error message comes back, and what happens if that's fol­lowed by a delayed duplicate that was slowed down in its response from a distant gateway when the flow control wouldn't take it but it worked its way back and got here late? What do you do then?' There would be this pause and they'd say, You've tried this!' And we'd reply, 'Hey, we never said that!'"

  Eventually they managed to communicate enough of Pup's architec­ture for it to become a cru
cial part of the ARPANET standard known as TCP/IP, which to this date is what enables data packets to pass gracefully across the global data network known as the Internet—with a capital "I." PARC's contribution is mostly unsung; one recent history of the Internet acknowledges only that PUP "inspired" TCP/IP. Metcalfe, Shoch, and the others have gotten used to their contribution being minimized unfairly. "The TCP/IP guys will never tell you they did this because of Xerox, because they don't remember it that way," Shoch said. "But we would sit there explaining the problems and trying to coach them along. We had all this shit up and running, and we couldn't tell them."

  Despite the lawyers' strictures on cooperating with ARPA and others on the outside, Shoch figured he should be allowed to play in his own sandbox. What was the point of having access to the Ethernet if you could not even use it to finish a paltry doctorate? He carefully assem­bled a stack of provisional clearances and approvals and made sure the Xerox bureaucracy knew that more was at stake for him than a ten- page paper in a scientific journal. Finally confident that he would be able to publish what he learned, he set out to pound on the Ethernet like a kid beating a drum.

  To begin with, he wrote a program that would spew message bits onto the Ethernet for a ten-minute period without interruption, start­ing at the stroke of midnight. He and a colleague named Jon Hupp spent hours laboriously loading the software into 100 idle Altos after dark, when the rest of the lab had gone home. Then they waited for 12:10 a.m. and repeated the circuit, this time collecting each machines data on how many of its packets got through to their destination, com­pared to how many got tied up in the system by gridlock. Before this ordeal had run its course, Shoch realized he had committed himself and his friend to the dreariest and stupidest exercise known to man.

  Doctorate or not, he was damned if he was going to go through that again. Somehow he had to figure out how to make the machines do the bleak scutwork. It should not be an insurmountable task, he figured— just a simple matter of getting the machines to tell him they were idle, then distributing by Ethernet the very same program that he had hauled around to a hundred offices on that first night. In principle this was not much different from booting up or sending files over the net, something that happened at PARC dozens of times a day.

  Within a few days Shoch had written a new program that would instruct one Alto to ask the others if they were free. The only complica­tion was that unoccupied Altos at PARC were never entirely idle. After their owners departed they remained on, spending the night running a diagnostic program that continually tested the memory chips—those old buggy Intel 1103s—for incipient failures. The test program, which kicked in automatically whenever the machine remained inactive for a certain number of minutes (just like todays automatic screen savers), would prevent the Altos from responding affirmatively to his query.

  Shoch strolled down the hall to David Boggs, who had written the diagnostic. "Could you just add this thing to the program so the machine will answer mine and tell me he's free?" he asked.

  Boggs shrugged. Helping a colleague like this was coin of the realm at PARC. "No problem," he said. "I can do that."

  "And while you're at it, can you make sure that if I send him this land of packet he'll reboot with my software?"

  "I can do that."

  "So all of a sudden," Shoch recalled later, "you could send out a 'Hello' to any free machine and say, 'I'm going to load this program on you through the net, and when I say "Go" you're to start generating bits onto the network. Then I'm going to poll you all remotely one by one and take back all the data and make a printout. And then I'm going to produce this chart and get my Ph.D.'"

  Inducing fifty or a hundred Altos to answer a summons dispatched by the press of a button gave him a giddy feeling of power. "It was like sitting at the helm of the Starship Enterprise," he said, "in control."

  Perhaps that was why his thoughts soon turned from communicating directly with each machine to instructing them to talk to each other. What if, rather than loading the same program onto fifty machines from one central point, he gave each one the ability to seek out the others?

  "Instead of my having to find fifty machines, maybe I just find two, and tell each of them to find fifty more. Then each of those machines is told to find twenty-five more." And on and on—the possibilities were infinite. "It's the kind of thing you think about late at night," he laughed—but it was not the first time something of the sort had occurred to the human mind. As he explained the concept one day to a group of colleagues, he was interrupted by a Stanford intern named Steve Weyer.

  "That's the tapeworm!" Weyer said.

  "The what?"

  "The tapeworm, from The Shockwave Rider. I'll give you a copy."

  The Shockwave Rider was a recent novel by the British science fiction writer John Brunner. In the course of the story its hero unleashes an invulnerable self-perpetuating virus—perhaps the first computer virus of fact or fiction—to destroy a sinister global network. ("It can't be killed. Not short of demolishing the net!") Paging through Weyer's copy of the book, Shoch thought the similarities between his device and Brunner's were mostly metaphorical. "It wasn't quite what we were doing, although it did evoke the right images," he recalled. He appropriated the name anyway, slightly truncated, and from that moment on Shoch's Ethernet- polling program was known around the building as "the worm."

  Rather than destroy the network like Brunner's hero, Shoch aimed to enhance it by allowing PARC's distributed machines to perform compu­tations in concert. Theoretically, the worm could be programmed to start out in one machine and reach out to others as its computing needs expanded. "In the middle of the night, such a program could mobilize hundreds of machines in one building," he wrote later. Before morning, as users arrived to reclaim their machines, the worm would retreat. After hibernating in a machine or two during daylight, it would re-emerge the next evening—an image that led one of Shoch's colleagues to liken it less to a worm than to a vampire.

  The point was to convert 200 Altos, each with the power of a single processor, into one distributed, interlinked machine with the exponential power of 200! Shoch's brainstorm hinted at a method of compounding processor power that would one day find wide application in the field of supercomputers. (The IBM machine "Deep Blue" employed a distantly related architecture when it instantaneously mustered billions of calcula­tions to defeat chess grandmaster Gary Kasparov in 1997.)

  Shoch was well aware that the privilege of taking over scores of idle computers in their owners' absence meant he would have to govern his program stringently. He invested his worm with the ability to seek out idle Altos, boot up a host machine through the network, and replicate itself by sending copies of itself from machine to machine. But he strictly forbade it access any Alto's disk drive—a necessary precaution lest it inadvertently overwrite someone's work, which he knew would be viewed as "a profoundly antisocial act."

  Even within those limits he reveled in finding new ways to extend and elaborate the worm. In time he improved the program to the point where it could remain in constant communication with its dispersed offspring, so it would know how many segments were out and what they were doing. The mother worm could even sense if an Alto had crashed with an active segment inside, a signal that she would need to colonize another host.

  But in imagining a world with a tapeworm gone amok, John Brunner had not underestimated the obstinacy of a runaway program. Eventu­ally the moment arrived for Shoch and his colleagues to discover just how stubborn and dangerous one could be. The worm was about to commit an "antisocial" act beyond anything they could have foreseen.

  One night they set a small worm loose to test a control function. Confi­dent they had loaded a perfectly innocuous program into a modest num­ber of Altos, they left it running and went home. At some point—they never figured out exactly when and why—one piece of the program became corrupted so badly that it crashed its host computer. Sensing it had lost a segment, the control worm sent out a tendri
l to another idle Alto. That host crashed, and the next, and the next. The program was now probing insatiably for new hosts and "killing" every one it touched. For hours the silent carnage spread through the building until scores of machines were disabled. Then morning came.

  Ted Kaehler was one of those who got to Iris office to find his machine in a state of catatonia. Without thinking twice, he rebooted. But it soon seemed that the entire building had been taken over by a malevolent spirit. Whenever a machine went idle, instead of slipping routinely into the memory diagnostic it would suffer this bizarre seizure and the. "We couldn't get rid of it," he recalled.

  Finally the alarm reached the worm researchers. It was still early in the morning when Boggs and Shoch responded, barely able to wipe the sleep from their eyes. They began disassembling the network and probing its recesses, like exterminators pursuing rats through a sewer line. Even after they were sure they had eradicated the program from every Alto, machines continued to crash bizarrely. Exhausted from the ordeal, Shoch found himself reimagining scenes from The Shockwave Rider—"workers running around the building, fruitlessly trying to chase the worm and stop it before it moves somewhere else."

  The building had to be harboring a reservoir of diseased worms still seeking hosts. Finally they realized where the nest was located. More than a dozen Altos were up on the third floor, the administrative quarters. No one up there had arrived for work yet, and the offices were all locked.