Architects of Intelligence

by Martin Ford

  MARTIN FORD: After that you went on to start Coursera with Daphne Koller, who is also interviewed in this book. Then you moved on to Baidu. Can you describe your path through those roles?

  ANDREW NG: Yes, I helped to start Coursera with Daphne because I wanted to scale online teaching both around AI and other things to millions of people around the world. I felt that the Google Brain team already had tremendous momentum at that point, so I was very happy to hand the reins over to Jeff Dean and move on to Coursera. I worked at building Coursera from the ground up for a couple of years until 2014 when I stepped away from my day-to-day work there to go and work at Baidu’s AI Group. Just as Google Brain helped transform Google into the AI company you perceive it to be today, the Baidu AI group did a lot of work to transform Baidu into the AI company that a lot of people now perceive Baidu to be. At Baidu, I built a team that built technology, supported existing business units, and then systematically initiated new businesses using AI.

  After three years there the team was running very well, so I decided to move on again this time becoming the CEO of Landing AI and a general partner at AI Fund.

  MARTIN FORD: You’ve been instrumental in transforming both Google and Baidu into AI-driven companies, and it sounds like now you want to scale that out and transform everything else. Is that your vision for AI Fund and Landing AI?

  ANDREW NG: Yes, I’m done transforming large web search engines, and now I’d rather go and transform some other industries. At Landing AI, I help to transform companies using AI. There are a lot of opportunities in AI for incumbent companies, so Landing AI is focused on helping those companies that already exist to transform and embrace those AI opportunities. AI Fund takes this a step further, looking at the opportunities for new startups and new businesses to be created from scratch built around AI technologies.

  These are very different models with different opportunities. For example, if you look at the recent major technological transformation of the internet, incumbent companies like Apple and Microsoft did a great job transforming themselves to be internet companies. However, you only have to look at how big the “startups,” like Google, Amazon, Baidu, and Facebook are now and how they did such a great job building incredibly valuable businesses based on the rise of the internet.

  With the rise of AI there will also be some incumbent companies, ironically many of them were startups in the previous age, like Google, Amazon, Facebook, and Baidu, that’ll do very well with the rise of AI. AI Fund is trying to create the new startup companies that leverage these new AI capabilities we have. We want to find or create the next Google or Facebook.

  MARTIN FORD: There are a lot of people who say that the incumbents like Google and Baidu are essentially unshakable because they have access to so much data, and that creates a barrier to entry for smaller companies. Do you think startups and smaller companies are going to struggle to get traction in the AI space?

  ANDREW NG: That data asset that the large search engines have definitely creates a highly defensible barrier to the web search business, but at the same time, it’s not obvious how web search clickstream data is useful for medical diagnosis or for manufacturing or for personalized educational tutors, for example.

  I think data is actually verticalized, so building a defensible business in one vertical can be done with a lot of data from that vertical. Just as electricity transformed multiple industries 100 years ago, AI will transform multiple industries, and I think that there is plenty of room for multiple companies to be very successful.

  MARTIN FORD: You mentioned AI Fund, which you founded recently and which I think operates differently from other venture capital funds. What is your vision for AI Fund, and how is it unique?

  ANDREW NG: Yes. AI Fund is extremely different from most venture capital funds, and I think most venture capital funds are in the business of trying to identify winners, while we’re in the business of creating winners. We build startups from scratch, and we tell entrepreneurs that if you already have a pitch deck, you’re probably at too late a stage for us.

  We bring in teams as employees and work with them, mentor them, and support them, whatever is needed to try and build a successful startup from scratch. We actually tell people that if you’re interested in working with us, don’t send us a pitch deck, send us a resume and then we’ll work together to flesh out the startup idea.

  MARTIN FORD: Do most people that come to you already have an idea, or do you help them come up with something?

  ANDREW NG: If they have an idea we’re happy to talk about it, but my team has a long list of ideas that we think are promising but we don’t have the bandwidth to invest in. When people join us, we’re very happy to share this long list of ideas with them to see which ones fit.

  MARTIN FORD: It sounds like your strategy is to attract AI talent in part by offering the opportunity and infrastructure to found a startup venture.

  ANDREW NG: Yes, building a successful AI company takes more than AI talent. We focus so much on the technology because it’s advancing so quickly, but building a strong AI team often needs a portfolio of different skills ranging from the tech, to the business strategy, to product, to marketing, to business development. Our role is building full stack teams that are able to build concrete business verticals. The technology is super important, but a startup is much more than technology.

  MARTIN FORD: So far, it seems that any AI startup that demonstrates real potential gets acquired by one of the huge tech firms. Do you think that eventually there’ll be AI startups that will go on to have IPOs and become public companies?

  ANDREW NG: I really hope there’ll be plenty of great AI startups that are not just acquired by much larger startups. Initial public offering as a tactic is not the goal, but I certainly hope that there’ll be many very successful AI startups that will end up thriving as standalone entities for a long time. We don’t really have a financial goal; the goal is to do something good in the world. I’d be really sad if every AI startup ends up being acquired by a bigger company, and I don’t think we’re headed there.

  MARTIN FORD: Lately, I’ve heard a number of people express the view that deep learning is over-hyped and might soon “hit a wall” in terms of continued progress. There have even been suggestions that a new AI Winter could be on the horizon. Do you think that’s a real risk? Could disillusionment lead to a big drop off in investment?

  ANDREW NG: No, I don’t think there’ll be another AI winter, but I do think there needs to be a reset of expectations about AGI. In the earlier AI winters, there was a lot of hype about technologies that ultimately did not really deliver. The technologies that were hyped were really not that useful, and the amount of value created by those earlier generations of technology was vastly less than expected. I think that’s what caused the AI winters.

  In the current era, if you look at the number of people actually working on deep learning projects to date, it’s much greater than six months ago, and six months ago, it was much greater than six months before that. The number of concrete projects in deep learning, the number of people researching it, the number of people learning it, and the number of companies being built on it means the amount of revenue being generated is actually growing very strongly.

  The fundamentals of the economics support continued investment in deep learning. Large companies are continuing to back deep learning strongly, and it’s not based on just hopes and dreams, it’s based on the results we’re already seeing. That will see confidence continue to grow. Now, I do think we need to reset the expectations about AI as a whole, and AGI in particular. I think the rise of deep learning was unfortunately coupled with false hopes and dreams of a sure path to achieving AGI, and I think that resetting everyone’s expectations about that would be very helpful.

  MARTIN FORD: So, aside from unrealistic expectations about AGI, do you think we will continue to see consistent progress with the use of deep learning in more narrow applications?

  ANDREW NG: I think there
are a lot of limitations to the current generation of AI. AI is a broad category, though, and I think when people discuss AI, what they really mean is the specific toolset of backpropagation, supervised learning, and neural networks. That is the most common piece of deep learning that people are working on right now.

  Of course, deep learning is limited, just like the internet is limited, and electricity is limited. Just because we invented electricity as a utility, it didn’t suddenly solve all of the problems of humanity. In the same way, backpropagation will not solve all the problems of humanity, but it is turning out to be incredibly valuable, and we’re nowhere near done building out all the things we could do with neural networks trained by backpropagation. We’re just in the early phases of figuring out the implications of even the current generation of technology.

  Sometimes, when I’m giving a talk about AI, the first thing I say is “AI is not magic, it can’t do everything.” I think it’s very strange that we live in a world where anyone even has to say sentences like that—that there’s a technology that cannot do everything.

  The huge problem that AI has had is what I call the communications problem. There’s been tremendous progress in narrow artificial intelligence and also real progress in artificial general intelligence, but both of these things are called AI. So, tremendous progress in economics and value through narrow artificial intelligence is rightly causing people to see that there’s tremendous progress in AI, but it’s also causing people to falsely reason that there’s tremendous progress in AGI as well. Frankly, I do not see much progress. Other than having faster computers and data, and progress at a very general level, I do not see specific progress toward AGI.

  MARTIN FORD: There seem to be two general camps with regard to the future of AI. Some people believe it will be neural networks all the way, while others think a hybrid approach that incorporates ideas from other areas, for example symbolic logic, will be required to achieve continued progress. What’s your view?

  ANDREW NG: I think it depends on whether you’re talking short term or long term. At Landing AI we use hybrids all the time to build solutions for industrial partners. There’s often a hybrid of deep learning tools together with, say, traditional computer vision tools because when your datasets are small, deep learning by itself isn’t always the best tool. Part of the skill of being an AI person is knowing when to use a hybrid and how to put everything together. That’s how we deliver tons of short-term useful applications.

  On balance, there’s been a shift from traditional tools toward deep learning, especially when you have a lot of data, but there are still plenty of problems in the world where you have only small datasets, and then the skill is in designing the hybrid and getting the right mix of techniques.

  I think in the long term, if we ever move toward more human-level intelligence, maybe not for AGI but more flexible learning algorithms, I think that we’ll continue to see a shift toward neural networks, but one of the most exciting things yet to be invented will be other algorithms that are much better than backpropagation. Just like alternating current power is incredibly limited, but also incredibly useful, I think backpropagation is also incredibly limited, but incredibly useful, and I don’t see any contradiction in those circumstances.

  MARTIN FORD: So, as far as you’re concerned, neural networks are clearly the best technology to take AI forward?

  ANDREW NG: I think that for the foreseeable future, neural networks will have a very central place in the AI world. I don’t see any candidates on the horizon for replacing neural networks, that’s not to say that there won’t be something on the horizon in the future.

  MARTIN FORD: I recently spoke with Judea Pearl, and he believes very strongly that AI needs a causal model in order to progress and that current AI research isn’t giving enough attention to that. How would you respond to that view?

  ANDREW NG: There are hundreds of different things that deep learning doesn’t do, and causality is one of them. There are other things, such as not doing explainability well enough; we need to sort out how to defend against adversarial attacks; we need to get a lot better at learning from small datasets rather than big datasets; we need to get much better at transfer or multitask learning; we need to figure out how to use unlabeled data better. So yes, there are a lot of things that backpropagation doesn’t do well, and again causality is one of them. When I look at the amount of high value projects being created, I don’t see causality as a hindering factor in them, but of course we’d love to make progress there. We’d love to make progress in all of those things I mentioned.

  MARTIN FORD: You mentioned adversarial attacks. I’ve seen research indicating that it is fairly easy to trick deep learning networks using manufactured data. Is that going to be a big problem as this technology becomes more prevalent?

  ANDREW NG: I think it is already a problem, especially in anti-fraud. When I was head of the Baidu AI team we were constantly fighting against fraudsters both attacking AI systems and using AI tools to commit fraud. This is not a futuristic thing. I’m not fighting that war right now, because I’m not leading an anti-fraud team, but I have led teams and you feel very adversarial and very zero-sum when you’re fighting against fraud. The fraudsters are very smart and very sophisticated, and just as we think multiple steps ahead, they think multiple steps ahead. As the technology evolves, the attacks and the defenses will both have to evolve. This is something that those of us shipping products in the AI community have been dealing with for a few years already.

  MARTIN FORD: What about privacy issues? In China especially, facial recognition technology is becoming ubiquitous. Do you think we run the risk that AI is going to be deployed to create an Orwellian surveillance state?

  ANDREW NG: I’m not an expert on that, so I’ll defer to others. One thing that I would say, is that one trend we see with many rises in technology is the potential for greater concentration of power. I think this is true of the internet, and this is true again with the rise of AI. It becomes possible for smaller and smaller groups to be more and more powerful. The concentration of power can happen at the level of corporations, where corporations with relatively few employees can have a bigger influence, or at the level of governments.

  The technology available to small groups is more powerful than ever before. For example, one of the risks of AI that we have already seen is the ability of a small group to influence the way very large numbers of people vote, and the implications of that on democracy is something that we need to pay close attention to, to make sure that democracy is able to defend itself so that votes are truly fair and representative of the interests of the population. What we saw in the recent US election was based more on internet technologies rather than AI technologies, but the opportunity is there. Before that, television had a huge effect on democracy and how people voted. As technology evolves, the nature and texture of governance and democracy changes, which is why we have to constantly refresh our commitment to protecting society from its abuse.

  MARTIN FORD: Let’s talk about one of the highest-profile applications of AI: self-driving cars. How far off are they really? Imagine you’re in a city and you’re going to call for a fully autonomous car that will take you from one random location to another. What’s the time frame for when you think that becomes a widely available service?

  ANDREW NG: I think that self-driving cars in geofenced regions will come relatively soon, possibly by the end of this year, but that self-driving cars in more general circumstances will be a long way off, possibly multiple decades.

  MARTIN FORD: By geofenced, you mean autonomous cars that are running essentially on virtual trolley tracks, or in other words only on routes that have been intensively mapped?

  ANDREW NG: Exactly! A while back I co-authored a Wired article talking about Train Terrain (https://www.wired.com/2016/03/self-driving-cars-wont-work-change-roads-attitudes/) about how I think self-driving cars might roll out. We’ll need infrastructure changes, and societal and legal change
s, before we’ll see mass adoption of self-driving cars.

  I have been fortunate to have seen the self-driving industry evolve for over 20 years now. As an undergraduate at Carnegie Mellon in the late ‘90s, I did a class with Dean Pomerleau working on their autonomous car project that steered the vehicle based an input video image. The technology was great, but it wasn’t ready for its time. Then at Stanford, I was a peripheral part of the DARPA Urban Challenge in 2007.

  We flew down to Victorville, and it was the first time I saw so many self-driving cars in the same place. The whole Stanford team were all fascinated for the first five minutes, watching all these cars zip around without drivers, and the surprising thing was that after five minutes, we acclimatized to it, and we turned our backs to it. We just chatted with each other while self-driving cars zipped passed us 10 meters away, and we weren’t paying attention. One thing that’s remarkable about humanity is how quickly we acclimatize to new technologies, and I feel that it’s not going to be too long before self-driving cars are no longer called self-driving cars, they’re just called cars.

  MARTIN FORD: I know you’re on the board of directors of the self-driving car company Drive.ai. Do you have an estimate for when their technology will be in general use?

  ANDREW NG: They’re driving round in Texas right now. Let’s see, what time is it? Someone’s just taken one and gone for lunch. The important thing is how mundane that is. Someone’s just gone out for lunch, like any normal day, and they’ve done it by getting in a self-driving car.

  MARTIN FORD: How do you feel about the progress you’ve seen in self-driving cars so far? How has it compared with your expectations?

  ANDREW NG: I don’t like hype, and I feel like a few companies have spoken publicly and described what I think of as unrealistic timelines about the adoption of self-driving cars. I think that self-driving cars will change transportation, and will make human life much better. However, I think that everyone having a realistic roadmap to self-driving cars is much better than having CEOs stand on stage and proclaim unrealistic timelines. I think the self-driving world is working toward more realistic programs for bringing the tech to market, and I think that’s a very good thing.