Solomon's Code

by Olaf Groth

  One might start to worry about how much data these Digital Barons have and how much they need to deliver the products and services we so readily consume. We already grant a remarkable amount of power in the data we share, and new AI models can amplify that power even more. Both generative adversarial networks (GANs) and one-shot learning systems increase the accuracy and precision of AI outputs. GANs pit two AI systems against each other, with one creating an artificial output—a fake image, for example—and the other comparing it with real examples in hopes of detecting the flaws. The competitive feedback loop between the two improves the accuracy of both systems collectively, like steel sharpening steel.

  The one-shot learning model could vastly expand the breadth, increase the speed, and decrease the cost of machine learning. With this approach, a system already trained to recognize a variety of objects can start to identify similar things based on just one or a few examples, not unlike the way a toddler learns to avoid a hot tea kettle after touching a hot pan on the stove. A one-shot learning system trained to identify dozens of different weapons in a battlefield scenario could learn to recognize a different threat based on just a few instances of it, building on what it has learned about the characteristics of the others.

  There’s one caveat to these approaches, however. While both GANs and one-shot learning models enable deep networks to learn from few examples, both need the networks to be primed with human-labeled examples in the first place. Absent a more efficient and practical way to generate labeled data across a wide variety of new domains—and, frankly, even if such a source emerges—there’s no reason for the Digital Barons to limit the flow of incoming data, and every incentive to expand it.

  THE CAMBRIAN COUNTRIES

  In recent years, Fei-Fei Li has become one of the AI world’s most well-known academic researchers, with her papers cited thousands of times and her position as director of the Stanford Artificial Intelligence Lab. Li traveled a long way to get there, moving with her parents from Beijing to New Jersey when she was sixteen years old before enrolling at Princeton University to study physics. She went on to earn her PhD from Cal Tech and later led the development of ImageNet, a massive online database that contains millions of hand-labeled images.

  That sort of data set seems almost routine today, but when Li first published ImageNet in 2009 it had emerged from a somewhat unusual notion—that a better algorithm couldn’t make better decisions on its own; it also needed better data. With such a large, labeled data set now available to them, AI researchers started competing against one another in the annual ImageNet Challenge, where they could see whose algorithm correctly identified the most objects in the millions of images. In 2010, the winning system won with an accuracy rate of 72 percent (compared with a 95 percent average for humans). By 2012, though, an algorithm entered by Geoff Hinton and his colleagues at the University of Toronto unexpectedly knocked the mark up to 85 percent.# Their innovation? An interesting new technique called “deep learning,” now one of the AI field’s bedrock models.

  These days, Li is still working to change the way the AI field thinks, not only about its technological challenges but about the people who solve them. She supplies one of the leading voices for efforts to increase diversity in the ranks of academics, developers, and researchers. As a computer science professor at Stanford and director the university’s AI Lab, Li regularly mentors students whose gender or ethnicity is vastly underrepresented in a field dominated by Caucasian and Asian men. “I’m focused on diversity, not just gender or race, but diversity of thought,” Li says. “I think we absolutely have to do it. I don’t think this issue can wait. This technology is going to shape humanity and if we don’t get the right representation of humanity to participate in this technology, we’ll have negative consequences.”**

  Yet, for all the parallels between Li’s professional standing and the broader AI ecosystem, none reflect as fundamental a force as her joint participation in academia and private-sector business, particularly in her role as chief scientist of Google Cloud AI. On sabbatical from Stanford during the spring of 2018, Li helped design products and services intended to extend AI capabilities to more businesses and individuals, with the idea of democratizing access to these advanced technologies. And much like her desire to facilitate greater diversity among the people involved in the field, she hopes to open AI development to people around the world. “Science knows no borders,” she says, and she hopes to build a tighter collaboration between US and Chinese AI research. In fact, in late 2017, she helped launch Google’s basic research lab in Beijing, her hometown.

  Despite the cultural, governmental, and economic contrasts between China and the United States, the rich ecosystem of development and cross-fertilization within and between each nation have set them apart from all others. The tight ties between academia and private-sector serve as one of the most critical drivers of AI development in these “Cambrian Countries,” with their unique combination of Digital Barons, leading academies, vibrant entrepreneurialism, and their cultural dynamism. Microsoft, Baidu, Facebook, Tencent, and other private sector companies have ponied up millions of dollars to attract the top academic talent in both nations, including Chinese companies landing students at US colleges and, to a lesser extent, vice versa. Those companies have put millions more into research initiatives at leading universities from Boston to Beijing. And while the same critical academic-business ties occur in other parts of the world, other regions haven’t developed the same depth of interdependency that’s found in China and the United States, which are well attached to each other’s hip.

  Shaoping Ma plays that bridging role at Tsinghua University, where he leads a joint research center with Sogou.com, the country’s second-largest search engine behind Baidu. Ma focuses on search and information retrieval. In China, Ma says through an interpreter, pioneering research remains well behind the United States, but its companies have done a comparable job of developing a variety of applications, such as machine translations. The massive consumer base and the data it produces create more opportunities for Chinese start-ups, he says, and close cooperation with industry helps academic developers gain access to those large volumes of real data to fuel and advance their research. “I don’t think we can overtake the US in five years in general,” he says, “but in some application areas we could.”

  These academic-corporate relationships are vital to AI development because each side brings a different set of motivations to development. Jeannette Wing moved from her post in charge of Microsoft’s global network of research labs to head up Columbia University’s Data Sciences Institute. For her, the chance to tackle the major, long-term, basic research problems in AI sciences proved too tempting after years in a mix of industry, academic, and government positions. She still works closely with the other sectors, now with an eye on the benefits of AI, but she and her colleagues in academia don’t have to worry about the same short-term, bottom-line pressures that companies face.

  So, while industry has two big advantages over academia—“big data and big compute”—Wing can try to solve deeper fundamental questions about AI models and related issues. “Only the academic community has the luxury of time to understand the science underlying the techniques,” she says. “And it’s important to have this understanding because they [AI technologies] are already in self-driving cars and they are already being used in the criminal justice system, etc. So, the end users—drivers, pedestrians, judges, defendants—are going to be affected by this technology. It behooves the scientific community to provide a fundamental understanding of how these techniques work.”

  THE CASTLE COUNTRIES

  Mikhail Burtsev wields the impressive scientific chops one might expect of Russian academia. He holds a PhD in computer science, which he earned while trying to model the evolution of human cognitive abilities. He focused on the theoretical side, trying to reconcile some Russian cybernetics ideas and develop a new way to get models of machine learning to interact with o
ne another. “I’m still doing experiments with living neurons and what’s happening with real neural networks [in the brain],” he says. “I have some theoretical part that stems from Russian neurophysiology, which is not really widely known in the West, and I’m thinking about how to incorporate those in the net architecture of AI agents.”

  Lately, though, Burtsev has shifted his expertise in new directions, creating a project called iPavlov, an initiative that simultaneously supports and belies some of the outside stereotypes about Russian artificial intelligence. On one hand, it’s a deeply academic effort backed by government funding, having earned a grant in 2017 from the National Technology Initiative 2035. On the other, Burtsev is using that funding to create an open-source platform and database that will help developers build better conversational AI systems, something especially difficult given the nuances of the Russian language. In other words, one Russian friend told us, the government is financing an open-source project that will help everyone create better artificial intelligence.

  “Russia has rather good potential in this field, but it’s not realized yet,” Burtsev says. “What we have is more or less good fundamental education and we have very good computer science skills for students, but on the other hand if you look at publication outcomes of AI, we’ll see Russia is somewhere like fortieth place in the world and it’s not very visible in the scientific landscape.” The Russian government and President Vladimir Putin have recognized artificial intelligence as a vital piece of future geopolitical power, security, and influence, and it has started pumping more public funding into the field. Added to the pool of existing private investment, the public support has helped expand the resources available to start-ups, Burtsev says, but the country has not come close to its full potential given its academic standards.

  The Castle Countries, which include Russia and Western Europe, have developed key expertise in certain facets of artificial intelligence, mainly on the academic side, hence the name “castle” in reference to the idiomatic “ivory tower.” But none of them have the Digital Barons on the commercial side, which oversee the free flow of ideas, technologies, and capital from academia to the private sector, nor the dynamism to break down their castle walls. In places like the United States, Japan, and China, established technology transfer programs and a more robust entrepreneurial infrastructure help facilitate this. In Russia, however, conversations with several entrepreneurs reveal a start-up ecosystem that gets little support, with large, often state-backed companies and banks limiting the critical flow of capital to new ventures and the transfer of technology from academia to the private sector. Less tolerance for business risk, which is always inherent in early-stage tech start-ups, creates a chicken-egg problem for some small businesses in Russia, says George Fomitchev, the founder of Endurance Robotics, which develops a variety of laser-etching systems, chatbot software, and interactive robots. Investors want to see more established track records of sales, preferably to large Russian companies, Fomitchev says, but the big companies won’t commit to products or services until they’re fully baked. For one of their customer service chatbots, Endurance had to use the metrics it generated from work with British American Tobacco to pitch to Burger King, which needs a different set of products and strategies.

  This leaves small start-ups like Endurance in a bind: Leave for other markets or keep trying to break into the established channels of capital, such as the oligarchies or the few accelerators and funding programs that exist. After Grigory Sapunov and his cofounder launched Intento, a platform that helps companies test different AI platforms and find the right fit for their operations, they realized they would need to break into other markets for sales and for the support their own business needed to grow. They have a marketable idea and have been building a client base. Using various public or proprietary data sets, they can test the many cloud-based AI platforms and find the ones best suited for a particular client. Then, their platform allows customers to use and switch between many of the providers, depending on the task, cost, or performance.

  “The main disadvantage is that Russia is very far from the modern centers of the AI movement,” Sapunov says. “A lot of Russian start-ups are trying to do something meaningful, but a lot of them are working for the local market, and it’s hard for them to get into the global market.” So, one of Intento’s cofounders set up shop at an accelerator in Berkeley, California, looking to tap into the US market and the pool of resources in the Bay Area. Too few people run their own businesses in Russia, Sapunov says, and the California base gives them a connection to the entrepreneurial spirit there. Back home, he tries to poach talent from Yandex and some of the other large high-tech companies in Russia. “There are lots of interesting jobs in start-ups, but people want to have a safe place,” he says. “It’s a problem because AI is a field that’s very dynamic. To catch the wave, you have to take some risk in your activities.”

  Western Europe exhibits some characteristics similar to Russia, including a somewhat wider gap between academia and industry than that in the United States and China. However, the European model—which combines a high degree of scientific and academic competence with its sizable manufacturing base and more open approach to data sharing—differs from Russia’s in two significant ways. First, Russia tends to channel more of its computer science and mathematics prowess into defense and national intelligence than do the EU member countries. Second, and perhaps more significant, Europe has developed a much more robust entrepreneurial ecosystem than their Russian counterparts. While its start-up environment trails that of the United States and China, Europe has seen the emergence of several key digital hubs in Berlin, Hamburg, London, and Paris. Tallinn, the capital of Estonia, has become a cybersecurity hub and one of the world’s most advanced centers of digital government. Gothenburg, Sweden, and Helsinki, Finland, have launched Nordic AI initiatives that facilitate open collaboration across academic and entrepreneurial sectors. Even some off-the-beaten-path places feature small but important pockets of innovation, including Lugano, Switzerland, a picturesque town of 60,000 people that’s home to AI pioneer Jürgen Schmidhuber and the Dalle Molle Institute for Artificial Intelligence Research.

  Still, Europe remains a long shot away from translating these pockets into formative and global economic powerhouses on the scale of the United States and China. Much of its current AI development tends to flow through manufacturing and other traditional industries. Add the European Union regulations on data privacy and security issues, and the innovative environment gets a bit more restricted (regardless of what one believes about the merits of those data-privacy rules). Damian Borth would like to see a more nuanced set of regulations, one that would keep protections where needed without stifling industry and start-up activity. Borth, director of the Deep Learning Competence Center at the German Research Center for Artificial Intelligence (DFKI), suggests four classifications. If an AI system has an impact on human life, it slots into the A category and is regulated more closely. If it can impact the environment but not directly threaten people, it slots into Class B, and so on toward less restriction for systems that can do less harm. “If you want to have same market size as the US, Borth says, “you have to go to all of Europe, and then you have to deal with all the regulatory systems.” So, for all the work done on artificial intelligence across the region, entrepreneurs still slip away to bigger markets.

  What Germany and most other European countries do well is “the boring stuff,” Borth jokes, like manufacturing. The game for personal data is lost. Starting a European version of Google, Amazon, or a continent-originated social network would certainly be an uphill battle (albeit one that the privacy-conscious part of the world might welcome) in a field in which “winner takes all” and the installed base of consumers plays a decisive role. So, AI development there tends to build around customer transaction data, data gleaned from the Internet of Things, and business-to-business applications.

  Still, Europe’s diversity of instituti
ons and funding sources, as well as an emerging EU effort to develop a digital commons for technological innovation, makes it an interesting alternative model. More institutions, companies, and governments across the continent are helping in that evolution. Borth and his colleagues at DFKI, for example, work closely with industry to get better access to data and help companies integrate new AI models into their processes, from business operations to human-friendly robotics in the factory. England has become home to critical outposts for some of the world’s Digital Barons, including DeepMind, which Google acquired for about $625 million (£400 million).†† And while it lacks homegrown Digital Barons and the sheer volume of entrepreneurial activity in the United States and China, London and some of England’s top universities have become rich seedbeds for entrepreneurial spinoffs, in many ways similar to the close academic-commercial relationships in Silicon Valley. From 2012 through the first half of 2016, the United States invested $18.2 billion in AI, China $2.6 billion, and the U.K. $850 million, according to a Goldman Sachs report.‡‡

  The very roots of AI reach back to Alan Turing and Bletchley Park, and universities such as Cambridge and Oxford already have established some of the world’s most prominent high-tech research centers that host renowned experts such as Nick Bostrom, Jaan Tallinn, and Huw Price. Concerns about AI’s effects on Britain’s economy and its people have permeated the government, as well. In spring 2018, the British government joined with private companies and investment firms to commit $1.4 billion toward AI development. The money will support several initiatives, including UK-focused investment funds, an AI supercomputer at the University of Cambridge, and a new Center for Data Ethics.