At present, publishers are pigeons, fed by feeding the horses, such as Google and Facebook, through many stomachs of intermediation. Like print publishers in the past, they earn profits not chiefly from content but from the time, indirectly measured, their readers give to ads. But Google and others are discovering that human attention can be depleted. As Eich remarks, it sinks away “until dopamine levels recover.” Already evident is an Internet epidemic of “banner blindness.”
Into this breach Eich is hurling his one billion BATs—the unit of exchange for an ingenious new decentralized open-source and efficient digital advertising platform based on Vitalik Buterin’s Ethereum blockchain. Advertisers award BATs to publishers based on the attention of users measured by the pattern of their usage. Users too will be paid in BATs for accepting ads that they want to see or choose to tolerate in exchange for micropayments. They can donate these BATs back to favored publishers or use them in exchange for content.
This transparent system keeps user data private while removing middlemen and delivering fewer but more relevant ads that users actually seek. Publishers gain a larger share of the returns while advertisers get better reporting and performance and customers get ads they explicitly accept for pay. Rather than manipulating the viewer to see ads, advertisers find viewers who are interested in their appeals. Eich concludes, “Brave will reset the online ad-based Web ecosystem, giving advertisers, publishers, and customers a win-win solution whose components and protocols can become future Web standards.”
As Eich told a TEDx audience in Vienna in October 2016, “Try to imagine a world in which you own your own dossier; it’s your own online life—you should own your own data. If you own it, then you can give terms of service [to the giant walled gardens on the Web], as well as them giving you their ‘terms of service’ that no one ever reads. . . . This would create a new web.”
In a world of abundant information but scarce time, what do people value most? As Kevin Kelly declares, “The only things that are increasing in cost while everything else heads to zero are human experiences. . . . Cheap abundant [virtual reality] will be an experience factory.”1
I first met Brendan Eich when he joined me on the advisory board of the Los Angeles-based startup OTOY. My entrance into what Kelly describes as the experience factory also came courtesy of OTOY. Its inventor-founder, Jules Urbach, has been turning computer models for 3D scenes into digital images that can be sent across the Net, shown on any screen, and experienced as real.
OTOY’s metaverse is something entirely new. Its virtual worlds will be almost indistinguishable for many purposes from the topology of the real world. As Kevin Kelly breathlessly describes it, “We’ll use it to visit environments too dangerous to risk in the flesh, such as war zones, deep seas, or volcanoes. Or we’ll use it for experiences we can’t easily get to as humans—to visit the inside of a stomach, the surface of a comet. Or to swap genders, or [as Jaron Lanier wants] become a lobster. Or to cheaply experience something expensive, like a flyby of the Himalayas.”2 Most importantly, we will use it for new social interactions in new settings—concerts, dances, theaters, stadia.
In this new virtual domain, how will ownership be identified and defended? Musical works, for example, combine contributions from composers, lyricists, performers, distributors, and other participants, all of whom have particular claims. As an OTOY white paper explains: “Blockchains can handle intricate property rights needed for complex digital assets that can be routinely copied and for which time-stamped proof of authorship is crucial. Tokens, on the other hand, enable immediate many-sided transactions executed from embedded contracts executed within the blockchain process.”
The key promise of OTOY, as John Carmack, CTO of Facebook/Oculus, observes, is an entirely new platform and user interface for the Internet, announced in 2015 at a joint press conference with Eich at the spectacular San Francisco headquarters of Autodesk, an OTOY investor.
Eich, then still at Mozilla, and OTOY announced that together they are delivering a next-generation OTOY ORBX video codec—the way images are coded for transmission and decoded onto the screen—that is ported to JavaScript and thus usable with any browser. It becomes ORBX.js and is embodied in OTOY software rather than in custom hardware chips that take up to ten years to specify and burn into silicon.
With OTOY’s ORBX rendering protocol translated into Eich’s JavaScript, any browser can blossom into a three-dimensional space. Users can escape confinement to a screen and occupy a space. No longer limited to typing on a screen, you can inscribe writing on the walls of the world—an advance that portends the end of the segmented, top-down, walled-garden, cookie-crumbling Internet. After his Autodesk press conference, Eich declared in his blog: “This morning I saw the future.”
Eich was particularly enthusiastic about replacing cumbersome digital rights management schemes with watermarking the video itself in every intra-frame. Ari Emanuel, agent supreme, the leader of William Morris Endeavor, and a crucial promoter of OTOY, believes that this advance might eventually eliminate the need for digital rights management altogether. Although this kind of per-user watermarking used to be prohibitively costly, OTOY can do it in the cloud for an estimated pennies per film.
As Jeff Kowalski, Autodesk’s CTO, points out, the benefits go beyond major cost reduction in computer-generated imaging and similar processing work. The OTOY software increases collaboration and innovation by freeing creative people from big workstations. The GPU cloud means many alternative ideas, camera angles, and such can be tried without waiting hours for each rendering. “Even from the beach,” he said, “on your 4G connected tablet.” Or in your contact lenses, newly sensory clothes, holographic visors, and immersive electronic spaces.
The fulfillment of Neal Stephenson’s Metaverse is possible for the first time. Without leaving their homes, people will be able to travel anywhere in the world, or even beyond it, and enjoy a full visual and even haptic experience. They will be able to interact with images and buy and sell rights within the virtual domains. They will attend plays and films and sports and news events and new kinds of morphic narratives. They will hang glide through the Alps, fly to the moon, and cruise beyond it to life after Google.
In this cause, OTOY is now allied with Disney, Unity, Facebook, HBO, Jon Stewart, NHL, Discover Channel, Autodesk, Nvidia, and Amazon.
As Eich tweeted in July 2017, OTOY is “predicting the holodeck/matrix/metaverse future by building it.”
CHAPTER 17
Yuanfen
When Stephen Balaban proposed to Betty Meng, the skies above blessed the event with a giant diamond ring. He had first glimpsed her across a crowded room at Oren’s Hummus Shop on University Avenue in Palo Alto four years earlier. Now, on the morning of August 21, 2017, the two were standing in Madras, Oregon, at an optimal point in the seventy-mile-wide “path of totality” of the solar eclipse.
The winter stars came out. An eclipse wind chilled the air. Two minutes and four seconds of daytime darkness passed. Eerie bands of shadow shimmered on the ground. Then, at exactly 10:29 a.m., the sun signaled its return. The expected diamond of sunlight burst out from behind the moon, and Stephen presented his girlfriend with a terrestrial version. The awed Betty said yes.
As Peter Thiel writes in Zero to One, “every great entrepreneur is first and foremost a designer. . . . ”1 But the designs do not always work on the first try. Balaban had bought that diamond ring on the crest of a sudden, unexpected, and growing wave of business success doing something no one, including himself, expected him to do: beat Google and Amazon at one of their own games. But a crucible of deep learning preceded this triumphal proposal scene.
Fluent in Mandarin, Balaban went to Beijing in 2010 as a college senior, taking a semester off from studying computer science and economics at the University of Michigan. In China, he helped found “a clone,” as he describes it, of the Y-Combinator startup accelerator. He named it for Yuan Fen, the Chinese concept of the fate that brings peop
le together. He had the educational experience of eventually watching the venture fizzle because of conflicts among the founders.
Returning to Michigan, he took his degree and headed for Silicon Valley—after Beijing, “the real deal.” He moved into a room in San Francisco’s historic Chinatown with its own sink and a shared bathroom down the hall, an hour by bicycle and Caltrain from Palo Alto. It was April 2012, and Balaban was starting a company to teach machines to see and learn, working on face recognition for mobile devices. He gave this company a Greek rather than a Chinese name—Lambda Labs—after Alonzo Church’s universal model of computation, an American version of a Turing machine.
In 2012, people knew that face recognition was coming to handsets, but no one had been able to make it compact and fast enough. Balaban’s work caught the attention of the academic image gurus Zak Stone and Nicolas Pinto at Perceptio Corporation, and they hired him in November to develop mobile face recognition technology for the iPhone.
Like all such projects by then, this one would be based on deep neural-network processing. But it was mobile machine learning, Balaban explains, “meaning running face recognition and other neural nets on the phone’s own graphics processing unit, not even uploading to the sky.” He saw that artificial intelligence did not need to take place in giant data warehouses. This was a contrarian insight worthy of a Thiel Fellow (and by mid-2013 he was living with two of them, Austin Russell and Thomas Sohmers), but it took some years before he capitalized on it. “Basically I was learning deep learning.”
He left Perceptio in November 2013. Two years later, Stone and Pinto sold the company to Apple for $200 million. Its face-recognition functions are now standard in new iPhones. Meanwhile, Balaban lured his fraternal twin brother, Michael, away from the growing success story NextDoor, which provides localized information and services, to be chief technology officer of Lambda Labs. Michael seems to have shared his twin’s impeccable timing—by 2015, NextDoor had become a unicorn worth more than a billion dollars.
The Balabans started working on hardware. Using portable AI and face recognition, they conceived a wearable camera embedded in a baseball cap, resembling Google Glass or Snapchat spectacles. The problem was that no one in Silicon Valley could build the “Lambda Hat” prototypes, so Stephen was back in China for six months of exploring the manufacturing hives of Shenzhen, across the bay from Hong Kong. He ended up with a cool hat, better Mandarin, and a sharper sales pitch but no manufacturer or market for the product. “The technology was not mature,” Stephen decided.
Despite the disappointment, he still did not want to work on “something that wasn’t mine.” In early 2015, Gary Bradski—the robotics pioneer who developed computer vision at Intel, founded the Willow Garage robotics incubator, which convinced Wired’s Kevin Kelly that “robots have wants,” and started Industrial Perception, which made “stevedore robots” that could, as Stephen Balaban described it, “pick up and chuck a box so elegantly” that Google bought them—that Gary Bradski—invited Balaban to join his deep-learning team at Magic Leap. Launched in 2010, the Google-funded virtual reality venture in Florida had heretofore raised half a billion dollars while generating more national magazine covers than virtual reality advances. Neal Stephenson had just joined the company (as chief futurist), but Balaban wasn’t convinced, however magic and well-funded the leap.
What direction to leap instead?
In July 2015, the house in Atherton that Balaban, Austin Russell, and Thomas Sohmers were renting finally sold for its $10 million asking price. At the same time, Sohmers caught the eye of Thiel’s prestigious Founders Fund. Thiel put up $2 million to tape-out Sohmers’s new chip at Taiwan Semiconductor Manufacturing Company. Meanwhile, venture money from Thiel, 1517, and other funds rolled in for Russell’s “stealth” driverless car project up at Pony Tracks. And Balaban found an unexpected direction.
The month before, Chris Olah, Vitalik Buterin’s high school friend who beat him to the Thiel Fellowship and was now an intern at Google Brain, published a blog post with two Google software engineers. It was titled “Inceptionism: Going Deeper into Neural Networks.”2
The name itself was multilayered—a reference to the neural net architecture they were using, which in turn was a reference to an Internet meme about “going deeper,” which in turn was a quotation from the 2010 Christopher Nolan film Inception, in which a thief tunnels through other people’s dreams. The blog post laconically presented “some simple techniques for peeking inside these [neural] networks” and then showed a series of increasingly trippy photos, as if the machine were hallucinating. A little gray kitten became the stuff of nightmares: a shaggy beast with forehead and haunches bubbling with dark dog eyes and noses.
To Balaban, the code and its results were a visual confirmation of what Yoshua Bengio, a colleague of Geoffrey Hinton in the Montreal crucible of AI, calls the “manifold learning hypothesis.” Bengio sees the essential job of a neural network as learning a hierarchy of representations in which each new layer is built up out of representations resolved in a previous layer. The machine begins with raw pixels and combines them into lines and curves transitioning from dark to light and then into geometrical shapes, which finally can be encoded into elements of human faces or other targeted figures. Scramble this process at an early stage and you get an artfully inflected picture; scramble it higher up in the hierarchy and you get a phantasia of “dream and nightmare” images, as Bengio puts it. In dreams and nightmares, as in machine-learning feedback loops, no new information is perceived. Without new inputs, the mind or machine churns the old images in intriguing but unresolved patterns.3
Balaban was one of hundreds of people who were captivated by Olah’s post. On July 1, Google released the code, now named “Deep Dream,” and coders leapt on the chance to make their own dream images.
Balaban himself set out to develop a deep learning-powered image editor for the general public, which he offered on a simple website with various filters, most with names extracted from either art (“charcoal,” “art deco”) or the psychedelic subculture (“salvia,” “self-transforming machine elves”).
It was less than two months before the 2015 Burning Man festival. The website “burners.me” discovered Balaban’s app, which he had named “Dreamscope,” and published a blog post referring to Philip K. Dick’s novel of slippery reality, Do Androids Dream of Electric Sheep? Thirteen vividly Dreamscoped Burning Man photographs followed, full of psychedelic eyes, mushrooming shaggy-dog faces, merging human chimeras, and iterated swirls.4
The Dreamscope app “took off faster than anything I had ever seen. . . . Millions of downloads in the first day,” remembers Balaban. “It was the first time people got a real peek into how neural networks could see the world.”
Then Stephen and Michael Balaban figured out how to support nearly one million users, each running his own little machine-learning gradient and editor. (Lambda Labs was still just the Balaban twins and their cousin.) Scaled up by a distributed queue-processing system, “it allowed us to add new nodes to the pool on demand.” The flaw was that all the GPUs were controlled by Amazon Web Services, which had to be paid.
Stephen Balaban was following the Google model of giving your product away and charging for “premium” subscriptions. The problem was that most of his “customers” found the free access to addictive psychedelic photo-paintings good enough. There were a hundred thousand takers for the $9.95 “premium edition,” but a million dollars wasn’t enough.
Within a few months, Dreamscope pretty much died of its own success. Amazon Web Service bills mounted to forty thousand dollars a month. The company still had $150,000 in the bank, but it was running out of runway. As Alexandra Wolfe documents in her rousing book on the initial class of Thiel Fellows, Valley of the Gods, not all their projects flourish, no matter how good their system of the world.
“That’s the time most startups don’t make it,” Stephen says. That is also the time, as Danielle Strachman stresses, tha
t the power of an entrepreneurial community comes into play. Balaban remembered appreciating that about Strachman and Gibson when he first encountered them: “Mike and Danielle recognize the emotional component of starting a company that a lot of people neglect—the emotional rollercoaster that is very draining. I observed that they were really good at making sure that everyone had support networks for when you are, in Elon Musk’s apt description, staring into the abyss and chewing on glass.”
Facing the ballooning Amazon Web Services bills, Balaban went back to 1517; Danielle and Mike supplied another $150,000. That extended the runway another four or five months. Not enough. Austin Russell invested twenty thousand dollars in it (he later added another hundred thousand), and so did Gary Bradski at Magic Leap, among others. Balaban managed to put together another half-million-dollar round.
At that point, he felt a visceral resistance to sending any more money to Amazon. He simply couldn’t do it. It was a “Zero to One” moment, defying the most settled consensus in the Valley, the assurance of venture capitalists that it is suicidal to compete with Amazon and Google by building infrastructure. This consensus was powerfully confirmed by two of the greatest success stories of the past decade, Netflix and Instagram, both of which scaled up to a valuation of scores of billions by using AWS. Balaban was told, “despite the cost, you focus on your users and scaling your business and let Amazon scale up the servers.”
Balaban, however, decided to quit AWS cold turkey. He spent sixty thousand dollars to build his own servers from scratch, influenced, perhaps, by Thomas Sohmers’s insight that present-day servers are kludges that waste 98 percent of their energy running data over wires to and from memory or sitting around in “wait states.” Balaban decided he had spent enough time and money on wait states. There had to be a better way than sending off terabytes of learning data over the Internet to a queue at the Amazon GPU farm. “It would be cheaper and faster,” he calculated, “to put them on disks and call Fed Ex.”
Life After Google Page 19