Where Wizards Stay Up Late: The Origins of the Internet

by Katie Hafner

  In 1983 the Defense Communications Agency decided that theARPANET had grown large enough that security was now a concern. The agency split the network into two parts: theMILNET, for sites carrying nonclassified military information, and theARPANETfor the computer research community. Before the split, there were 113 nodes in the combined network. Afterward, 45 nodes remained with theARPANET, and the rest went toMILNET. Administratively and operationally there were two different networks, but with gateways connecting them users couldn’t tell. The oldARPANEThad become a full-fledged Internet.

  In 1988, five years after the 1983ARPANETtransition to TCP/IP, the ISO finally produced standards for Open Systems Interconnection, and the U.S. Government immediately adopted the rival OSI protocols as its official standard. It appeared that OSI might prevail over TCP/IP. In Europe, where national governments decree the standards, it seemed an article of faith that OSI was the solution.

  On the other hand, an American culture of the Internet was growing exponentially, and its foundation was TCP/IP. And while governments throughout Europe were anointing OSI, something of an underground movement sprang up at European universities to implement TCP/IP.

  One key development in determining the outcome between TCP/IP and OSI turned out to be the popularity of theUNIXoperating system, which had been developed at AT&T’s Bell Laboratories in 1969.

  Programmers likedUNIXfor two primary reasons: Its flexibility let them tailor it to whatever program they were working on, and it was “portable,” meaning it could be made to work on many different computers. In the late 1970s, programmers at Berkeley developed their own brand ofUNIX, and seeded the computer science community with it. BerkeleyUNIXeventually became a fixture at universities and research institutions all over the world. Around 1981, Bill Joy, aUNIXhacker at Berkeley, got ARPA funding to write TCP/IP into a version of BerkeleyUNIX. BBN had already written a version ofUNIXwith TCP/IP, but Joy didn’t like it and decided to do it his own way.

  Then, in 1982, Joy joined a couple of Stanford Business School graduates who were starting a new company to build and sell powerful “workstations,” computers that were of an order of magnitude more powerful than personal computers. Joy was brought in as theUNIX expert. They called their company Sun (for Stanford University Network) Microsystems. The first Sun machines were shipped with the Berkeley version ofUNIX, complete with TCP/IP. BerkeleyUNIXwith TCP/IP would be crucial to the growth of the Internet. When Sun included network software as part of every machine it sold and didn’t charge separately for it, networking exploded.

  It further mushroomed because of Ethernet. While packet radio andSATNETsparked the thinking about a conceptual framework for internetworking, they were largely experimental. Ethernet—the local area network designed by Bob Metcalfe and his colleagues at Xerox PARC back in 1973—was a practical solution to the problem of how to tie computers together, either on a campus or at a company. Xerox began selling Ethernet as a commercial product in 1980. At around the same time, Bob Taylor’s division at Xerox PARC gave a grant to major research universities in the form of Ethernet equipment, powerful computers, and laser printers. It amounted to millions of dollars worth of hardware. Then a small networking company called Ungermann-Bass sold Ethernet as a connection between terminals and host computers. And Metcalfe started his own company, 3Com, to sell Ethernet for commercial computers, including Sun machines.

  Throughout the early 1980s, local area networks were the rage. Every university hooked its workstations to local area networks. Rather than connect to a single large computer, universities wanted to connect their entire local area network—or LAN—to theARPANET.

  Ethernet made this possible. Ethernets were simple and, compared to the 50-kilobit lines of theARPANET, they were tremendously powerful. Their rapid growth in the university and research community pushed the demand for network interconnection. If your whole university was not connected to theARPANET,CSNET gave you a way to connect one computer at your university to theARPANET. But it was Ethernet that created a huge networking constituency.

  At major research universities there would be a network of hundreds of computers that could all talk to each other over an Ethernet network. To send traffic from an Ethernet in say, San Diego, to another Ethernet in Buffalo, you sent it through theARPANEThub. In this way, theARPANETwas the centerpiece of what was called the ARPA Internet. And through the first half of the 1980s, the ARPA Internet resembled a star, with various networks surrounding theARPANET at the center.

  Perhaps what TCP/IP had to recommend it most was the fact that it was unerringly “open.” Its entire design was an open process, following a path first blazed by Steve Crocker and the Network Working Group and continuing into the Internet. TheARPANET, and later the Internet, grew as much from the free availability of software and documentation as from anything else. (By contrast, Digital Equipment’sDECNETwas a proprietary network.) The Internet also supported a wide range of network technologies. Although the satellite and packet-radio networks had finite lifetimes, they helped open developers’ eyes to the need to handle a multitude of different networks.

  Reforming e-mail

  The TCP and IP standards weren’t the only major renovation to networking in the early 1980s. For years, every e-mail program written for theARPANEThad depended on the original file-transfer protocol to serve as its barge for schlepping the mail back and forth. It may have been a neat hack to attach the mail commands to the file-transfer protocol at first, but the processing of e-mail had grown more complicated. In a message to his colleagues in the MsgGroup mailing list one day in late August 1982, Postel said, “If you really go look at the FTP spec, you will see that the mail commands are really some sort of wart.” Postel and a lot of others felt it was time to build a completely separate transfer mechanism for mail.

  Since the network was undergoing massive rearrangement anyway with the switch to TCP/IP, this seemed an appropriate time to bring out the new standard. Postel and his colleagues called it the simple mail transfer protocol (SMTP). It clarified existing practices, while adding a few new control features.

  At the same time, the growth of the network gave rise to a new problem. “When we got to about two thousand hosts, that’s when things really started to come apart,” said Craig Partridge, a programmer at BBN. “Instead of having one big mainframe with twenty thousand people on it, suddenly we were getting inundated with individual machines.” Every host machine had a given name, “and everyone wanted to be named Frodo,” Partridge recalled.

  Sorting out the Frodos of the Internet wasn’t unlike sorting out the Joneses of Cleveland or the Smiths of Smithville. Where one lived, precisely, was important in differentiating who one was. For years, sorting this out was among the most troublesome, messiest issues for the Internet, until at last a group chiseled out a workable scheme, called the domain name system, or DNS.

  The core of the DNS team was Jon Postel and Paul Mockapetris at ISI, and BBN’s Craig Partridge. They spent three months working out the details of the new addressing scheme and in November 1983 came forward with two RFCs describing the domain name system. “DNS was a very significant change in the way we thought about the system being organized,” said Postel. “Tree-branching” was the guiding metaphor. Each address would have a hierarchical structure. From the trunk to the branches, and outward to the leaves, every address would include levels of information representing, in progression, a smaller, more specific part of the network address.

  But that sparked a debate about the sequence of the hierarchy; what should come first or last. Postel and others finally decided on a specific-to-general addressing scheme. The Internet community also argued back and forth over what to name the domains, delaying any implementation for about a year. It was asserted by some, unconvincingly, that domain names should reflect specific funding sources—MIT, DARPA, for example. Eventually, a committee agreed on seven “top-level” domains: edu, com, gov, mil, net, org, and int. Now there could be seven Frodos: a com
puter named Frodo at a university (edu), one at a government site (gov), a company (com), a military site (mil), a nonprofit organization (org), a network service provider (net), or an international treaty entity (int).

  DARPA began pressuring people to adopt DNS addresses in 1985. In January 1986 a grand summit meeting took place on the West Coast, bringing together representatives of all the major networks. By the time the summit was over, everyone had agreed that yes, they really believed in the DNS concept. “And yes, here was how we were going to make it work,” Partridge recalled, “And yes, we have the technology to make it all fly.”

  Pulling the Plug

  The first hint Cerf got that the Internet was going to be embraced by a world outside the scientific and academic communities came in 1989, when he walked on to the exhibition floor at Interop, a trade show started by Dan Lynch in 1986 to promote interconnectivity through TCP/IP. In its first couple of years, Interop was attended by a few hundred hardcore networking people. By 1989 the show was teeming with men and women in business attire. “It was an epiphany to walk into Interop and see the major money being spent on exhibitions with huge demonstrations set up,” Cerf said. “I realized, oh my God, people are spending serious money on this.” The exhibitors had names like Novell, Synoptics, and Network General. “We started looking at the network statistics and realized we had a rocket on our hands.” For years Cerf had seen the Internet as a successful, satisfying experiment. Occasionally he had hoped the Internet might reach a wider world of users. Now here was evidence that it was doing just that.

  By this time, virtually everyone was using TCP/IP. And there was an ever-increasing infrastructure built upon TCP/IP in Europe. TCP/IP was so widespread and so many people depended on it, that taking it down and starting over seemed unthinkable. By virtue of its quiet momentum, TCP/IP had prevailed over the official OSI standard. Its success provided an object lesson in technology and how it advances. “Standards should be discovered, not decreed,” said one computer scientist in the TCP/IP faction. Seldom has it worked any other way.

  By the late 1980s the Internet was no longer a star with theARPANET its center; it was a mesh, much like theARPANETitself. TheNSFNET program had democratized networks as evenCSNEThadn’t. Now anyone on a college campus with an Internet connection could become an Internet user. TheNSFNETwas fast becoming the Internet’s spine, running on lines that were more than twenty-five times faster thanARPANETlines. Users now had a choice between connecting to theARPANETor to theNSFNETbackbone. Many chose the latter, not only for its speed but because it was so much easier to connect to.

  As the 1990s approached, the number of computers in the world that were connected to one another via theNSFNETfar outstripped the number of computers connected to one another via theARPANET. TheARPANETwas now just one of hundreds of ARPA Internet networks, and a dinosaur, unable to evolve as quickly as the rest of the Internet.

  Bob Kahn, DARPA’s sole remaining champion of networking, had left the agency in 1985 to form the Corporation for National Research Initiatives, a nonprofit company whose charter was to foster research and development for a “national information infrastructure.” The people now running DARPA weren’t particularly interested in networking. In their view, all the interesting problems had been solved. Moreover, the agency was distracted by President Ronald Reagan’s Star Wars program. TheARPANETitself, which cost ARPA $14 million a year to run, looked arthritic next to the higher-speedNSFNET. DARPA management decided theARPANEThad outlived its usefulness. It was time to shut it down.

  Mark Pullen, a DARPA program manager who now ran the networking project, was given the task of decommissioning theARPANET . Exactly who gave the order from within DARPA’s higher reaches was never made quite clear. “No one wanted to be the ghoul that turned off theARPANET,” Pullen said, “so I became the source of the policy.” Pullen’s plan was to pull sites off theARPANETand put them on theNSFNETbackbone.

  It was hard telling Bob Kahn about the plan to decommission the network. Kahn had hired Pullen, and now Pullen played the executioner. “I had a sense he might feel I was turning off his greatest achievement,” Pullen said. “The one that seemed to hurt him worse was when I turned off the oldSATNET.”SATNET was slow and expensive and antiquated. “No doubt he must have felt it was his very own child. For valid reasons. But after he thought about it, he agreed I was doing the right thing.” (As it turned out, the money DARPA saved by turning off theARPANEThelped fund Kahn’s new project.)

  One by one, Pullen turned off the IMPs and TIPs that still lay at the heart of the original network. There was a certain sadness in its demise that called to mind the scene from Arthur C. Clarke’s2001:A Space Odysseywhere the fictional fifth-generation computer HAL is threatening its mission and has to be dismantled circuit by circuit. As HAL gradually loses its “mind,” it makes pathetic appeals for its “life” to Dave, the astronaut, who is doing the dismantling.

  In the case of theARPANET, the network died but its pieces lived on. “It wasn’t all that different from the breakup of Ma Bell,” Pullen recalled. “It involved locating clusters ofARPANETsites and finding someone to take them over.” In most cases, Pullen transferred eachARPANET site to one of the regional networks, and eased the transition by subsidizing the cost for a while. With the exception of two sites that went on to theMILNET, all the sites went to one or another of the regional networks. “I never had anyone object all that loudly,” Pullen said. “I think they all knew the time had come.” One site at a time, Pullen found new homes for them. Where there wasn’t a home, DARPA and NSF helped create one. SeveralARPANETsites in Southern California quickly formed their own regional network and called it Los Nettos; it was run by Danny Cohen and Jon Postel. The IMPs themselves were powered down, uncabled, and shipped away. Most were simply junked. Others went into service on theMILNET . The Computer Museum in Boston got one, and Len Klein-rock put IMP Number One on display for visitors at UCLA. The last IMP to go was at the University of Maryland. By coincidence, Trusted Information Systems, a company in Maryland where Steve Crocker now worked, was connected to that IMP. Crocker had been there at the birth and he was there at the death.

  By the end of 1989, theARPANETwas gone. TheNSFNETand the regional networks it had spawned became the principal backbone. That year, to mark both theARPANET’s twentieth anniversary and its passing, UCLA sponsored a symposium and called it “Act One.”

  In his speech, Danny Cohen found a source of inspiration, and he said this:

  “In the beginning ARPA created theARPANET. “And theARPANETwas without form and void. “And darkness was upon the deep.

  “And the spirit of ARPA moved upon the face of the network and ARPA said, ‘Let there be a protocol,’ and there was a protocol. And ARPA saw that it was good. “And ARPA said, ‘Let there be more protocols,’and it was so. And ARPA saw that it was good.

  “And ARPA said, ‘Let there be more networks,’and it was so.”

  Epilogue

  September 1994

  The party was BBN’s idea: gather a couple of dozen key players in Boston and celebrate the twenty-fifth anniversary of the installation of the firstARPANETnode at UCLA. By now, the Internet had grown far beyond a research experiment. As more people discovered its utility, it was becoming a household word. The Net promised to be to the twenty-first century what the telephone had been to the twentieth. Its existence was already reaching into nearly every aspect of American culture—from publishing to socializing. For many, e-mail had become an indispensable part of daily life. Housebound seniors used it to find companionship; some far-flung families used it as their glue. More people by the day were logging-on to conduct business or find entertainment on the Net. Analysts pronounced the Internet the next great marketing opportunity.

  The takeoff was just beginning. In 1990, the World Wide Web, a multimedia branch of the Internet, had been created by researchers at CERN, the European Laboratory for Particle Physics near Geneva. Using Tim Berners-Lee’s HTTP protocol, c
omputer scientists around the world began making the Internet easier to navigate with point-andclick programs. These browsers were modeled after Berners-Lee’s original, and usually based on the CERN code library. One browser in particular, called Mosaic, created in 1993 by a couple of students at University of Illinois, would help popularize the Web and therefore the Net as no software tool had yet done.

  The Net of the 1970s had long since been supplanted by something at once more sophisticated and more unwieldy. Yet in dozens of ways, the Net of 1994 still reflected the personalities and proclivities of those who built it. Larry Roberts kept laying pieces of the foundation to the great big rambling house that became the Internet. Frank Heart’s pragmatic attitude toward technical invention—build it, throw it out on the Net, and fix it if it breaks—permeated Net sensibility for years afterward. Openness in the protocol process started with Steve Crocker’s first RFC for the Network Working Group, and continued into the Internet. While at DARPA, Bob Kahn made a conspicuous choice to maintain openness. Vint Cerf gave the Net its civility. And the creators of the Net still ran the Internet Society and attended meetings of the Internet Engineering Task Force.

  Just as the party plans got under way, BBN got a new chief executive officer. George Conrades, a high-powered marketing veteran from IBM, had been recruited by BBN’s chairman Steve Levy to reshape the company’s businesses. Conrades loved the party idea. He seized on it as a perfect marketing vehicle. Conrades was smitten with BBN’s pioneering role. BBN wastheoriginal Internet company, he decided, a claim to fame the firm had yet to exploit. Make the party big and lavish. Rent out the Copley Plaza Hotel. Celebrate the network pioneers as if they had been the first to tread on the moon’s surface. Invite computer industry luminaries. And invite the press.