by Steve Levy
This listing showed the power of PageRank. It made BackRub much more useful than the results you’d get from the commercial search engines. Their list of institutions for the “university” query seemed totally random. The number one result for that generic term in AltaVista would give you the Oregon Center for Optics. Page recalls a conversation back then with an AltaVista engineer who told him that with the way pages were scored, a query for “university” was likely to get a page where that word appeared twice in the headline. “That doesn’t make any sense,” Page said, noting that such a search was more likely to get a minor university with redundancy in its title.
“If you want major universities, you should type ‘major universities,’” said the engineer. Page was appalled. “I’m like, well, they teach you in human computer interaction, which is my branch, that the user is never wrong. The person in the system is never wrong.”
Until that moment, the task of compiling a list of universities and ranking them in significance had been complicated, intellectually challenging, and labor-intensive. Some magazines employed large teams working for months to do just that. If you were to try to teach a computer to do that, your instinct would be to feed it data about SAT scores, graduation rates, prizewinners among faculty, and a thousand other factors. Then you’d have to figure out how to weigh them. The odds were low that a machine would crank out a rating that squared with the gut feeling of a well-educated citizen. But BackRub knew nothing about those statistics. It just knew how to take advantage of the fact that links created by the web community had implicitly produced a ranking that was better than any group of magazine editors or knowledge curators could come up with. Larry Page and Sergey Brin had figured out how to mine that knowledge before the information retrieval establishment and commercial search engines even realized that it existed.
“The whole field had suffered blinders,” says the computer scientist Amit Singhal, then a Bell Labs researcher who had been a protégé of Jerry Salton. “In some sense, search really did need two people who were never tainted by people like me to come up with that shake-up.”
Larry Page was not the only person in 1996 who realized that exploiting the link structure of the web would lead to a dramatically more powerful way to find information. In the summer of that year, a young computer scientist named Jon Kleinberg arrived in California to spend a yearlong postdoctoral fellowship at IBM’s research center in Almaden, on the southern edge of San Jose. With a new PhD from MIT, he had already accepted a tenure-track job in the CS department at Cornell University.
Kleinberg decided to look at web search. The commercial operations didn’t seem effective enough and were further hobbled by spam. AltaVista’s results in particular were becoming less useful because websites had gamed it by “word stuffing”—inserting multiple repetitions of desirable keywords, often in invisible text at the bottom of the web page. “The recurring refrain,” says Kleinberg, “was that search doesn’t work.” But he had an intuition of a more effective approach. “One thing that was not being used at all was the fact that the web was a network,” he says. “You could find people saying in the academic papers that links ought to be taken advantage of, but by 1996 it still hadn’t been.”
Kleinberg began to play around with ways to analyze links. Since he didn’t have the assistance, the resources, the time, or the inclination, he didn’t attempt to index the entire web for his link analysis. Instead he did a kind of prewash. He typed a query into AltaVista, took the first two hundred results, and then used that subset for his own search.
Interestingly, the best results for the query were often not included in those AltaVista solutions. For instance, if you typed in “newspaper,” Alta-Vista would not give you links for The New York Times or The Washington Post. “That’s not surprising, because AltaVista is about matching strings, and unless The New York Times happened to say, ‘I’m a newspaper!’ AltaVista is not going to find it,” Kleinberg explains. But, he suspected, he’d have more luck if he checked out what those 200 sites pointed to. “Among those 200 people who were saying ‘newspapers,’ someone was going to point to The New York Times,” he says. “In fact, a bunch of people were going to point to The New York Times, because among those 200 pages were some people who really liked to collect links for newspapers on the web. If you pulled in those links, and got a set of 5,000 to 10,000 of them, in a sense, you’d have a vote. The winner would be the one with the most in-links from the group.” It was the same lightbulb that had brightened over Larry Page’s head.
Sometime in December 1996, Kleinberg got the balance right. One of his favorite queries was “Olympics.” The summer games had been held in Atlanta that year, and there were thousands of sites that in some way dealt with the athletic contests, the politics, the bomb that a domestic terrorist had planted. The AltaVista results for that keyword were riddled with spam and were generally useless. But Kleinberg’s top result was the official Olympics site.
Kleinberg began showing his breakthrough around IBM. His managers quickly put him in touch with the patent lawyers. Most people took a look at what Kleinberg had set up and wanted him to find stuff for them. Even the patent attorney wanted Kleinberg to help him find sources for his hobby, medieval siege devices. By February 1997, he says, “all sorts of IBM vice presidents were trooping through Almaden to look at demos of this thing and trying to think about what they could do with it.” Ultimately, the answer was … not much. IBM was a $70 billion business, and it was hard to see how a research project about links on this World Wide Web could make a difference. Kleinberg shrugged it off. He was going to teach computer science at Cornell.
Through mutual friends at Stanford, Kleinberg heard about Larry Page’s project, and in July 1997 they met at Page’s office in the Gates Building. Kleinberg was impressed with BackRub. “In academia, when there’s a hard problem everyone wants to solve, you’re always implicitly competing with the other people who are working on it,” says Kleinberg. But neither mentioned that issue. Kleinberg encouraged Page to publish his findings, but Page wasn’t receptive. “Larry was worried about writing a paper,” says Kleinberg. “He was wary because he wanted to see how far he could get with it while he refined it.”
Kleinberg could see that his goals were different from Page’s. “They wanted to crawl the whole web and get it on racks of servers that they would accumulate,” Kleinberg says. “My view was ‘How can I solve this problem without having to sink three months into indexing the web?’ We had the same core idea, but how we went about it was almost diametrically opposite.” Kleinberg was trying to understand network behavior. Page and Brin were building something. “Kleinberg had this notion of authority, where your page can become good just by linking to the right pages,” says Page. “Whereas what I was doing was more of a traffic simulation, which is actually how people might search the web.”
Kleinberg kept up with Google. He turned down job feelers in 1999 and again in 2000. He was happy at Cornell. He’d win teaching awards and a MacArthur fellowship. He led the life in academia he’d set out to lead, and not becoming a billionaire didn’t seem to bother him.
There was yet a third person with the idea, a Chinese engineer named Yanhong (Robin) Li. In 1987, he began his studies at Beijing University, an institution that claimed prominence in the country by way of a metric: The Science Citation Index, which ranked scientific papers by the number of other papers that cited them. The index was used in China to rank universities. “Beijing University, measured by the number of citations its professors got from their papers, was ranked number one,” said Li.
Li came to the United States in 1991 to get a master’s degree at SUNY Buffalo, and in 1994 took a job at IDD Information Services in Scotch Plains, New Jersey, a division of Dow Jones. Part of his job was improving information retrieval processes. He tried the search engines at the time—AltaVista, Excite, Lycos—and found them ineffectual and spam-ridden. One day in April 1996 he was at an academic conference. Bored by the pr
esentation, he began to ponder how search engines could be improved. He realized that the Science Citation Index phenomenon could be applied to the Internet. The hypertext link could be regarded as a citation! “When I returned home, I started to write this down and realized it was revolutionary,” he says. He devised a search approach that calculated relevance from both the frequency of links and the content of anchor text. He called his system RankDex.
When he described his scheme to his boss at Dow Jones, urging the company to apply for a patent, he was at first encouraged, then disappointed when nothing happened. “So a couple of months later, I decided to write the application by myself.” He bought a self-help book on patent applications and filed his in June 1996. But when he told his boss, Dow Jones reasserted itself and hired a lawyer to review the patent, which it refiled in February 1997. (Stanford University would not file its patent for Larry Page’s PageRank system until January 1998.) Nonetheless, Dow Jones did nothing with Li’s system. “I tried to convince them it was important, but their business had nothing to do with Internet search, so they didn’t care,” he says.
Robin Li quit and joined the West Coast search company called Info-seek. In 1999, Disney bought the company and soon thereafter Li returned to China. It was there in Beijing that he would later meet—and compete with—Larry Page and Sergey Brin.
Page and Brin had launched their project as a stepping-stone to possible dissertations. But it was inevitable that they began to eye their creation as something that could make them money. The Stanford CS program was as much a corporate incubator as an academic institution. David Cheriton, one of the professors, once put it this way: “The unfair advantage that Stanford has over any other place in the known universe is that we’re surrounded by Silicon Valley.” It was not uncommon for its professors to straddle both worlds, maintaining posts in the department while playing in the high-tech scrum of start-ups striving for the big score. There was even a joke that faculty members couldn’t get tenure until they started a company.
Cheriton himself was a prime example of how the Stanford network launched companies and enriched the founders. One of the earlier gold strikes from Stanford was the founding of Sun Microsystems by a group that included Andy Bechtolsheim, Vinod Khosla, and Bill Joy. Cheriton was close to Bechtolsheim, so in 1995, when the latter decided to start Granite Systems, a networking start-up, the two collaborated. Eighteen months later, Cisco bought the company for $220 million.
Sergey Brin, Rollerblading his way around the corridors of Gates Hall, took notice. Though Brin and Page didn’t have classes with Cheriton, they headed to his office for some advice. They specifically wanted to know how they might interest a company into using PageRank in its own search technology. Cheriton told them that it would be difficult—Sun Microsystems, he reminded them, had been started out of frustration when companies had spurned Bechtolsheim’s attempts to sell his workstation technology.
Yet Brin and Page were reluctant at that point to strike out on their own. They had both headed to Stanford intending to become PhDs like their dads.
But licensing their search engine wasn’t easy. Though Brin and Page had a good meeting with Yahoo founders Jerry Yang and David Filo, former Stanford students, Yahoo didn’t see the need to buy search engine technology. They also met with an AltaVista designer, who seemed interested in BackRub. But the wise men back in DEC headquarters in Maynard, Massachusetts, nixed the idea. Not Invented Here.
Maybe the closest Page and Brin came to a deal was with Excite, a search-based company that had begun—just like Yahoo—with a bunch of sharp Stanford kids whose company was called Architext before the venture capitalists (VCs) got their hands on it and degeekified the name. Terry Winograd, Sergey’s adviser, accompanied them to a meeting with Vinod Khosla, the venture capitalist who had funded Excite.
That led to a meeting with Excite’s founders, Joe Kraus and Graham Spencer, at Fuki Sushi, a Palo Alto restaurant. Larry insisted that the whole BackRub team come along. “He always likes to have more people on his side than the opposite side, to get the upper hand,” says Scott Hassan, who attended along with Page, Brin, and Alan Steremberg. “They sent two people, so we had four.” The Excite people began comparison tests with BackRub, plugging in search queries such as “Bob Marley.” The results were a lot better than Excite’s.
Larry Page laid out an elaborate plan, which he described in detail in emails to Khosla in January 1997. Excite would buy BackRub, and then Larry alone would go to work there. Excite’s adoption of BackRub technology, he claimed, would boost its traffic by 10 percent. Extrapolating that in terms of increased ad revenue, Excite would take in $130,000 more every day, for a total of $47 million in a year. Page envisioned his tenure at Excite lasting for seven months, long enough to help the company implement the search engine. Then he would leave, in time for the fall 1997 Stanford semester, resuming his progress toward a doctorate. Excite’s total outlay would be $1.6 million, including $300,000 to Stanford for the license, a $200,000 salary, a $400,000 bonus for implementing it within three months, and $700,000 in Excite stock. (Since Page and Brin were working for Stanford while developing their work, the school owned the PageRank patent. Stanford would commonly make financial arrangements so that such inventors could hold exclusive licenses to the intellectual property they created. Eventually Stanford did so with Google, in exchange for 1.8 million shares.) “With my help,” wrote the not-quite-twenty-four-year-old student, “this technology will give Excite a substantial advantage and will propel it to a market leadership position.”
Khosla made a tentative counteroffer of $750,000 total. But the deal never happened. Hassan recalls a key meeting that might have sunk it. Though Excite had been started by a group of Stanford geeks very much like Larry and Sergey, its venture capital funders had demanded they hire “adult supervision,” the condescending term used when brainy geeks are pushed aside as top executives and replaced by someone more experienced and mature, someone who could wear a suit without looking as though he were attending his Bar Mitzvah. The new CEO was George Bell, a former Times Mirror magazine executive. Years later, Hassan would still laugh when he described the meeting between the BackRub team and Bell. When the team got to Bell’s office, it fired up BackRub in one window and Excite in the other for a bake-off.
The first query they tested was “Internet.” According to Hassan, Excite’s first results were Chinese web pages where the English word “Internet” stood out among a jumble of Chinese characters. Then the team typed “Internet” into BackRub. The first two results delivered pages that told you how to use browsers. It was exactly the kind of helpful result that would most likely satisfy someone who made the query.
Bell was visibly upset. The Stanford product was too good. If Excite were to host a search engine that instantly gave people information they sought, he explained, the users would leave the site instantly. Since his ad revenue came from people staying on the site—“stickiness” was the most desired metric in websites at the time—using BackRub’s technology would be counterproductive. “He told us he wanted Excite’s search engine to be 80 percent as good as the other search engines,” says Hassan. And we were like, “Wow, these guys don’t know what they’re talking about.”
Hassan says that he urged Larry and Sergey right then, in early 1997, to leave Stanford and start a company. “Everybody else was doing it,” he says. “I saw Hotmail and Netscape doing really well. Money was flowing into the Valley. So I said to them, ‘The search engine is the idea. We should do this.’ They didn’t think so. Larry and Sergey were both very adamant that they could build this search engine at Stanford.”
“We weren’t … in an entrepreneurial frame of mind back then,” Sergey later said.
Hassan quit the project. He got a job with a new company called Alexa and worked part-time on a start-up called eGroups. In fact, Larry and Sergey—this was before they had gotten a dollar in funding for Google—pitched in $5,000 each to help him buy computers for
eGroups. (The investment paid off less than three years later when Yahoo bought eGroups for an estimated $413 million.)
But for the next year and a half, all the companies they approached turned them down. “We couldn’t get anyone interested,” says Page. “We did get offers, but they weren’t for much money. So we said, ‘Whatever,’ and went back to Stanford to work on it some more. It wasn’t like we wanted a lot of money, but we wanted the stuff to get really used. And they would want us to work there and we’d ask, ‘Do we really want to work for this company?’ These companies weren’t going to focus on search—they were becoming portals. They didn’t understand search, and they weren’t technology people.”
In September 1997, Page and Brin renamed BackRub to something they hoped would be suitable for a business. They gave serious consideration to “The Whatbox,” until they realized that it sounded too much like “wetbox,” which wasn’t family-friendly. Then Page’s dorm roommate suggested they call it “googol.” The word was a mathematical term referring to the number 1 followed by 100 zeros. Sometimes the word “googolplex” was used generically to refer to an insanely large number. “The name reflected the scale of what we were doing,” Brin explained a few years later. “It actually became a better choice of name later on, because now we have billions of pages and images and groups and documents, and hundreds of millions of searches a day.” Page misspelled the word, which was just as well since the Internet address for the correct spelling was already taken. “Google” was available. “It was easy to type and memorable,” says Page.
One night, using a new open-source graphics program called GIMP, Sergey designed the home page, spelling the new company name in different colors, making a logo that resembled something made from children’s blocks. It conveyed a sense of amiable whimsy. He put an exclamation point after the name, just like Yahoo, another Internet company founded by two Stanford PhD dropouts. “He wanted it to be playful and young,” says Page. Unlike a lot of other web pages, the Google home page was so sparse it looked unfinished. The page had a box to type in requests and two buttons underneath, one for search and another labeled I’m Feeling Lucky, a startling bid of confidence that implied that, unlike the competition, Google was capable of nailing your request on the first try. (There was another reason for the button. “The point of I’m Feeling Lucky was to replace the domain name system for navigation,” Page said in 2002. Both Page and Brin hoped that instead of guessing what was the address of their web destination, they’d just “go to Google.”) The next day Brin ran around the CS department at Stanford, showing off his GIMP creation. “He was asking everybody whether it made any sense to put other stuff on the page,” says Dennis Allison, a Stanford CS lecturer. “And everybody said no.” That was fine with Page and Brin. The more stuff on the page, the slower it would run, and both of them, especially Page, believed that speed was of the essence when it came to pleasing users. Page later found it humorous that people praised the design for its Zen-like use of white space. “The minimalism is that we didn’t have a webmaster and had to do it ourselves,” he says.