by Katie Hafner
In the late 1960s, growing political unrest—both violent and nonviolent—caught the U.S. military by surprise. Army intelligence knew all about Prague, Berlin, and Moscow, but now the Pentagon was contemplating Newark, Detroit, and Chicago. The Army gathered information from dozens of U.S. cities on the location of police and fire stations, hospitals, and so forth. Someone in the Pentagon thought it would be a good idea to keep track of local troublemakers as well.
In 1972 public outcry erupted over the Army’s information gathering, and the order went out for the files to be destroyed immediately. But three years later, allegations surfaced that Army intelligence officers had instead used theARPANETto move the files to a new location. When the story broke, the fact that something like theARPANETexisted was news to most Americans. That the story was reported in the most draconian, cloak-anddagger terms only added to the stormy reaction. The result was a Senate investigation in which DARPA was called upon to explain how it was using theARPANET .
DARPA eventually proved that the Army files had not moved on theARPANETby reviewing hundreds of rolls of Teletype printouts that had been stored in a dusty crawl space at BBN. DARPA was vindicated, but a perceived entanglement with the Army’s clandestine operations was the last thing theARPANETneeded.
Changes at DARPA
Discussions about how DARPA would ultimately divest operational responsibility for the network had already begun around 1971. DARPA had set out to link the core processing capabilities in America’s top computer science research centers, and as far as the agency was now concerned, it had accomplished that. Its mission was research. It wasn’t supposed to be in the business of operating a network. Now that the system was up and running, it was becoming a drain on other priorities. It was time for DARPA to shed the service provider’s role.
Handling the transition was a touchy matter. TheARPANETwas now a valuable tool, and Roberts’s goal was to ensure its continued development. He commissioned several studies to help determine the best option. The best route, it seemed, was to maintain a networking research effort, but sell off the network itself to a private contractor. But sell to whom? The market for data-communications networks was still largely uncharted, and the big communications companies remained as skeptical as ever about the DARPA technology.
When Roberts contacted AT&T to see if it wanted to take over theARPANET , AT&T formed a committee of corporate and Bell Labs staff and studied the idea for months. AT&T could have owned the network as a monopoly service, but in the end declined. “They finally concluded that the packet technology was incompatible with the AT&T network,” Roberts said.
Others were not so blind to the prospects of computer networking. In July 1972 three engineers left BBN to form a company that would market a commercial network called Packet Communications Incorporated. And BBN itself had spoken to Roberts about buying the network and setting up a subsidiary company to operate it. Small, specialized, commercial carriers like these were the obvious solution to DARPA’s problem.
But soon Roberts had a new problem. In early 1973, BBN recruited him to run a new subsidiary calledTELENET(not to be confused with Telnet, the program for remote logins), which would market a private packet-switching service. Now unable to recommend a sale by the government toTELENET, Roberts arranged to have
theARPANETtransferred temporarily to the Defense Communications Agency—the same agency that Baran had refused to let build his network ten years earlier. The generals and majors and captains were still only slightly more receptive to the idea of a packet-switched network than AT&T, but Roberts expected this to be only an interim arrangement.
Having decided to accept the post atTELENET, Roberts now had the job of finding a successor. DARPA’s Information Processing Techniques Office, however, no longer held the appeal it once had for those in academe. Roberts approached a couple of his principal investigators at universities, but people who had active research programs did not want to leave them. Others were concerned about the salary cuts they would have to take to come to the DARPA office.
When Licklider heard of the trouble Roberts was having, he offered to return if Roberts needed him. Roberts knew it was just a gesture on Licklider’s part. Lick was now happily entrenched back at MIT. But after six months of searching, Roberts decided he didn’t have a choice. When Roberts called Lick’s office at MIT, he was told that Lick was on a walking tour in England. Roberts tracked him down in the middle of Wales and asked him if he was serious about taking the job. Licklider said yes. “I never saw Larry so overjoyed as when he finally got his replacement, because he was ready to go,” recalled a colleague of Roberts. “I think some of his delight was also the fact that Lick was going to take it, because he liked him. Everybody did.”
One of the first problems Lick faced upon his return was an awkward one involving BBN, his onetime employer. BBN was refusing to release the IMP source code—the original operating program written by the IMP Guys five years earlier.
BBN’s apparent impulse to control every aspect of the network created a certain tension from the beginning. Len Kleinrock and his group at UCLA’s Network Measurement Center had found it particularly frustrating. The center’s job was to find problems in the network, but when they did, BBN refused to help. “Whenever we found a software bug or inefficiency, we’d alert Heart specifically, and the response we got was typically almost a stiff arm,” said Kleinrock. “He’d say, ‘Look, the network is working, I’ve got a network to keep up and running and we’ll put your comment in the queue.’ We couldn’t fix it ourselves because we didn’t have the source code.”
The intellectual property issue finally boiled over when the engineers who left BBN to start their own company made a pointed request for their former employer to turn over the IMP source code. When BBN refused, they appealed to DARPA. While keeping source code proprietary is usually the prerogative of the company that develops it, the IMP source code was different, for it had been developed by BBN with federal funds. Moreover, BBN was in the midst of starting up theTELENETsubsidiary, which would be competing with the company started by the engineers. There was some concern at DARPA that if anyone in Congress or the press corps latched onto the fact that BBN’s subsidiary and no one else had access to the Defense Department–sponsored IMP technology, the agency could have a major problem on its hands.
Frank Heart and Dave Walden at BBN argued that since the source code was changed frequently to improve performance or fix bugs, the company was uncomfortable with distributing software that would become obsolete. The company held its ground.
Steve Crocker, by then a DARPA program manager who oversaw most of BBN’s contracts, took charge of the situation. He had control of about $6 million worth of work a year at BBN, about a quarter of the firm’s gross revenues. “I seriously considered moving all of the work we were supporting at BBN to other places because we couldn’t reach closure with them on the data rights to the IMP code,” he said. And he let BBN know what he was thinking.
Lick had had a long association with BBN, and he had high regard for people there, but he was dismayed by their posture. Reaction throughout the computer community was much the same. Finally, in direct response to Crocker’s threat, BBN agreed to provide the code to whoever asked for it, charging a nominal handling fee. “This was just an early version of much more serious intellectual property rights issues that emerged across the industry over the next few decades,” Crocker said.
With Bob Kahn’s help, Licklider also finished the job of spinning off the network to the Defense Communications Agency. Roberts’s earlier probes into selling off the network notwithstanding, federal rules required that a resource as rich as theARPANETcouldn’t just be sold to an outside party until the Defense Department determined whether it had a need for the Net in Defense. The agency eventually decided it did. In the summer of 1975,DCA took over the network management job from DARPA. Now DCA set operational policy for the network. DCA decided such things as where and when new nodes woul
d be installed and what the configuration of data lines should be. And BBN retained the contract for network operations, which meant the company carried out the decisions made by DCA. Shortly after the transition had been made, Lick returned to MIT, where he would spend the remainder of his career.
Soon everyone at BBN noticed an increasing number of forms that had to be filled out, even for the smallest of jobs. The DCA bureaucracy rattled many of the university people, too. “The agency generated a blizzard of memoranda from colonels and generals about things you were and weren’t allowed to do,” recalled Brian Reid. A few months after DCA took over theARPANET, several of the graduate students in Stanford’s computer science department came to a Halloween party dressed as colonels.
Relieved of the day-to-day management chores, DARPA was now able to concentrate on developing the newCATENETprotocols. By 1975, Yogen Dalal, a Stanford graduate student, had polished the Transmission-Control Protocol from Cerf and Kahn’s 1974 paper into a set of concrete specifications. The TCP specification was sent to three separate places for concurrent implementation: BBN, Cerf’s computer lab at Stanford, and University College, London.
At about the same time, Kahn and Steve Crocker began talking to Cerf about taking a job as a DARPA program manager in Washington. “I flew there and landed in a big snowstorm and thought, I don’t want to live in this part of the country,” Cerf recalled. “So I said no. But the real reason was I was afraid I wouldn’t do a very good job. I thought it would be a very visible position and if I screwed up, everybody would know.”
A year later, Kahn and Crocker tried again. This time Cerf accepted. “I was getting a little tired of being so fragmented at Stanford. I couldn’t get any research done. So I thought, Why not go to DARPA and have a bigger impact, because instead of the small little budgets I got at Stanford for my work, at DARPA I’d have an opportunity to have a bigger impact with more money to spend.”
As a program manager, Cerf was given responsibility for the packet radio, packet satellite, and research programs in what was now simply called the ARPA Internet. Cerf also continued working intensively on refining the TCP specification. A milestone occurred in October 1977, when Cerf and Kahn and a dozen or so others demonstrated the first threenetwork system with packet radio, theARPANET , andSATNET, all functioning in concert. Messages traveled from the San Francisco Bay area through a packet-radio net, then theARPANET, and then a dedicated satellite link to London, back across the packetsatellite network and across theARPANETagain, and finally to the University of Southern California’s Information Sciences Institute (ISI) in Marina del Rey. The packets traveled 94,000 miles without dropping a single bit.
During a break from a meeting Cerf chaired at ISI to discuss TCP in early 1978, Cerf, Postel, and Danny Cohen, a colleague of Postel’s at ISI, got into a discussion in a hallway. “We were drawing diagrams on a big piece of cardboard that we leaned up against the wall in the hallway,” Postel recalled. When the meeting resumed, the trio presented an idea to the group: break off the piece of the Transmission-Control Protocol that deals with routing packets and form a separate Internet Protocol, or IP.
After the split, TCP would be responsible for breaking up messages into datagrams, reassembling them at the other end, detecting errors, resending anything that got lost, and putting packets back in the right order. The Internet Protocol, or IP, would be responsible for routing individual datagrams.
“I remember having a general guideline about what went into IP versus what was in TCP,” Postel recalled. “The rule was ‘Do the gateways need this information in order to move the packet?’If not, then that information does not go in IP.” This twist on the protocol was inspired by a group of engineers from the Xerox Corporation’s Palo Alto Research Center who had attended the Stanford seminar. The Xerox team had solved similar issues on a smaller, proprietary scale by defining a family of protocols called the PARC Universal Packet, or PUP. Once Postel decided that creating a separate protocol was the right thing to do, he set about making sure it got done. With a clean separation of the protocols, it was now possible to build fast and relatively inexpensive gateways, which would in turn fuel the growth of internetworking. By 1978, TCP had officially become TCP/IP.
ETHERNET
In 1973, just when Cerf and Kahn had begun collaborating on the concept of internetworking, Bob Metcalfe at Xerox PARC was inventing the technological underpinnings for a new kind of network. Called a short-distance, or local-area, network, Metcalfe’s network would connect computers not in different cities but in different rooms.
Metcalfe had received his undergraduate degrees in electrical engineering and management from MIT and enrolled at Harvard for graduate school. But he hated Harvard immediately. “Harvard is full of old-money people,” he said. “MIT is full of no money. It was a class thing.”
Metcalfe took a job at MIT, where he felt more comfortable. The Institute was about to join theARPANET, and he was assigned to build the interface between MIT’s PDP-10 and its IMP. Harvard had a PDP-10, too, and Metcalfe offered to make a duplicate interface to give to Harvard. But the networking people at Harvard declined the offer. “They said they couldn’t possibly let a graduate student do something that important,” said Metcalfe. Harvard officials decided to have BBN do it. BBN in turn gave the job to its graduate student in residence, Ben Barker, who recruited John McQuillan, a fellow Harvard graduate student, to help him.
Although enrolled at Harvard, Metcalfe stayed at MIT to work on his dissertation, an examination of theARPANET. When Metcalfe submitted the work, Harvard rejected the finished dissertation as insufficiently theoretical (too much engineering and not enough science, he was told). The rejection was embarrassing for Metcalfe, who had just taken a job at Xerox Corporation’s Palo Alto Research Center after persuading his wife to leave her job to join him. He went to Xerox PARC anyway and began looking for a more theoretical topic for his thesis.
Then in 1972, while on DARPA-related business for PARC, Metcalfe stayed at the Washington home of his friend Steve Crocker. Crocker had gathered together some of the best technical people from the early sites to assist new sites and called these people “network facilitators.” Metcalfe, who had become something of anARPANET expert at MIT, was one of them.
Crocker had just been visited by Norm Abramson, the primary architect of theALOHAnetwork at the University of Hawaii. On the night of Metcalfe’s visit, Crocker left one of Abramson’s papers about theALOHAnetwork out. Metcalfe picked it up and read it before going to sleep. The paper kept him up much of the night. “The math in the paper was not only familiar but infuriating,” he said, “because they made the typical inaccurate assumptions that people make so that their models work.”
Metcalfe’s chance encounter with Abramson’s paper was to change his life. “I set out to make a new model for theALOHAsystem.” Within a few weeks, he was on a Xeroxfunded trip to the University of Hawaii. He stayed a month, and before coming home had added an extensive analysis of theALOHAsystem to his thesis. It was just the theoretical boost the dissertation needed. When he resubmitted the work, it was accepted.
But theALOHAsystem gave Metcalfe far more than a doctorate. Xerox PARC was in the process of developing one of the first personal computers, called the Alto. The company saw that customers would want to connect the machines, so Metcalfe, one of the resident networking experts, was assigned the task of tying the Altos together. Without a storeand-forward model, preventing data packets from colliding was impossible. But simply scaling down theARPANET by building a subnet of IMP-like store-and-forward computers would have been prohibitively expensive for a system designed to work in an office building.
Metcalfe had an idea, borrowed directly from theALOHANET—in fact, he would call it the Alto Aloha Network: Let the data packets collide, then retransmit at a random time. But Metcalfe’s idea differed in several respects from the Hawaiian system. For one thing, his network would be a thousand times faster thanALOHANET. It would also include
collision detection. But perhaps most important, Metcalfe’s network would be hardwired, running not by radio waves but on cables connecting computers in different rooms, or among clusters of buildings.
One computer wishing to send a data packet to another machine—say, a desktop workstation sending to a printer—listens for traffic on the cable. If the computer detects conflicting transmissions, it waits, usually for a few thousandths of a second. When the cable is quiet, the computer begins transmitting its packet. If, during the transmission, it detects a collision, it stops and waits before trying again—usually a few hundred microseconds. In both instances, the computer chooses the delay randomly, minimizing the possibility of retrying at the same instant selected by whatever device sent the signal that caused the collision. As the network gets busier, computers back off and retry over longer random intervals. This keeps the process efficient and the channel intact.
“Imagine you’re at a party and several people are standing around having a conversation,” said Butler Lampson, who helped Metcalfe develop the idea, describing the system. “One person stops talking and somebody else wants to talk. Well, there’s no guarantee that only one person wants to talk; perhaps several do. It’s not uncommon for two people to start talking at once. But what typically happens? Usually they both stop, there’s a bit of hesitation, and then one starts up again.”