by Robert Litan
Google has been conducting just these kinds of experiments for more than a decade, making small tweaks here and there to its website results and other aspects of its business. The company does all this with a growing army of statisticians picking, in each case, whether the A or the B design prompts the best reaction from website visitors, or clicks of ads, or so on. Indeed, Google runs so many A/B tests simultaneously that almost everyone who visits the site is probably being tested whether they know it or not. Put another way, there is no single Google website, but many different ones being tested at the same time. Other companies actively using A/B tests include Amazon, Netflix, and eBay.12
A semi-science has since grown up around the proper construction and administration of A/B tests, given that many other firms, whether Internet-based or still grounded in the physical world but with an Internet presence, use them. One well-known company providing a free tool for anyone or any firm that wants to conduct its own A/B test is Optimizely, founded by two ex-Google employees, Dan Siroker and Pete Koomen, who honed the testing technique for the Obama presidential campaign in 2012. An entrepreneur and writer, Eric Ries, has written the bible for A/B testing by tech startups, The Lean Startup. Ries and his book have even encouraged a whole lean-startup movement.13
Not all A/B-type tests are run on the Internet, because most commerce, after all, is still conducted in bricks-and-mortar facilities. One of the leading companies facilitating experiments in that context is Applied Prediction Technologies, founded by James Manzi (see the following box). Since its founding in 1999, APT has grown to the point where, according to Manzi, 30 to 40 percent of the largest U.S. retailers, hotels, restaurant chains, and retail banks in the United States had used or were using APT’s experimentation platform as of 2012.14
James Manzi: Analytical Pioneer
Although he is not a formally trained economist, he has the analytical tools and certainly acts like one, though in a business setting. Manzi backed his way into the analytics and experimentation business by accident, like many entrepreneurs with other businesses.15
Manzi studied math and science in his undergraduate years at MIT, with the intention of going on to get his PhD in either mathematics or physics. But after a year in graduate school at the University of Pennsylvania, Manzi realized he was better suited for a business career, one where he could use his considerable analytical skills.
He gained his first job at AT&T’s research labs, and shortly thereafter, at the age of 23 joined a newly formed consulting firm, Strategic Planning Associates (SPA). SPA specialized in applying analytical techniques to help companies determine which of their many lines of business offered the most opportunities and how to maximize profits pursuing them. Manzi used a variety of mathematical techniques, drawing on his earlier training, in multiple consulting assignments with the company.
With this experience under his belt, Manzi took the entrepreneurial plunge in 1999, launching with two other partners Applied Predictive Technologies. As noted in the text, APT has become a major force in enabling bricks-and-mortar businesses to conduct real-world experiments.
Innovation and Entrepreneurship: Experimentation as the Foundation of Growth
There is one other connection between experimentation and economics I cannot resist closing this chapter with, and it relates to entrepreneurs, many of whom are experimenters by nature, and whose importance to the economy I believe cannot be overstated. With the exception of a few economists like the great Joseph Schumpeter, Israel Kirzner, and a famous economist I am about to reveal, this was not generally well recognized among mainstream economists (and still isn’t in some quarters) until relatively recently. So this is a case where business has been out in front of economics, but economists are catching up and have begun to provide practical research that even the most skeptical entrepreneurs will find useful.
The reason I can make these claims is that I spent nearly a decade of my life on the periphery of entrepreneurship, but at the center of entrepreneurship research when I directed research for the Kauffman Foundation, the world’s largest foundation devoted primarily to advancing entrepreneurship. During the near decade I was there (2003 to 2012), Kauffman greatly expanded its support of mainstream economists to study various aspects of entrepreneurship, and in particular, its connection to innovation and economic growth. The list is long, but a sample of the notable economists who were funded included William Baumol of NYU (profiled in Chapter 4), Edmund Phelps of Columbia University (profiled below), Amar Bhidé of Tufts University (who had already made his mark in the field with a landmark study of innovative entrepreneurs),16 Josh Lerner of Harvard Business School, Steven Kaplan of the University of Chicago, and Alicia Robb, formerly of the Fed staff and at this writing a senior fellow at Kauffman itself.
I benefited greatly from frequent interactions with these remarkable individuals and many others, as well as from reading their research when writing two books on entrepreneurship and economic growth, one with an economist profiled earlier in this book (William Baumol) and the president of the Kauffman Foundation, Carl Schramm,17 and the sequel which Schramm and I coauthored.18 I learned more from writing these books, all the while enjoying the process, than from any other books I had previously written (until this one!). Although the books are very different (and of course I encourage readers of this book to read them), they have a common theme: Entrepreneurs, defined as those who commercialize new products, services, or ways of producing or delivering them, are among the most important, and arguably the most important, drivers of innovation and growth in average living standards.
Edmund Phelps
One sign that mainstream economists are taking entrepreneurship and its crucial connection with innovation more seriously is the shifting research agenda by one of the profession’s leading macroeconomists, Edmund “Ned” Phelps, who won the Nobel Prize in 2006. Through much of his career, Phelps’ research focused on inflation and unemployment, and the theory of economic growth.
Phelps’ latest book, Mass Flourishing, makes the case for entrepreneur-driven innovation as one of the major drivers of economic growth, but worries about various impediments to such innovation today.19 His book extends his earlier work emphasizing the importance of cultural factors that influence entrepreneurial propensities.
Phelps is from Illinois, went to Amherst for undergraduate studies, and earned his PhD at Yale University. During the course of his prolific academic career, Phelps has held positions at the RAND Corporation, Yale, the University of Pennsylvania, and Columbia, where he cofounded the Center on Capitalism and Society.
On a personal note and with full disclosure: Although I saw him from afar when I was a lowly research assistant at the Brookings Institution in the 1970s and witnessed his presentation or discussion of papers at the semiannual meetings of the Brookings Papers on Economic Activity, I got to know Phelps much later in life (for both of us), while I was at the Kauffman Foundation. The Foundation supported Phelps’ work and the launch of his Center, and thus played a part in encouraging Phelps’ research and writing about entrepreneurship and innovation during the post-Nobel phase of his career.
There are many ways to classify such innovative entrepreneurs. The distinction I believe to be most relevant to this chapter, and thus this book, is between those who are convinced they know better what consumers eventually will want than consumers themselves and are highly unlikely to change their minds––think Steve Jobs, Larry Page, and Sergey Brin––and those entrepreneurs who start with one idea (typically written in detail in a business plan), but often quickly shift to another after some initial encounters with consumers and the marketplace. The latter category includes the entrepreneurial equivalents of scientists, those who have a hypothesis (the business plan) about how a business can be profitable, and then test it through refinements in the products or services they sell to consumers. These experimental entrepreneurs keep experimenting until they get it right.20
I know of no studies that attempt
to determine what portion of successful entrepreneurs are inspirational and those that are experimental, but my own impression is that most fall into the second camp or, at the very least, have both elements as part of their nature. As a matter of logic this stands to reason, since I cannot count how many times I have heard the adage, from economists and businessmen (and women), that what makes an entrepreneur different is that he or she sees opportunity where others see only problems. This is a variation of the advice I have heard many successful entrepreneurs give to aspiring entrepreneurs: Channel your entrepreneurial passions into fixing, for commercial gain, the problems that you personally have found to be highly irritating or frustrating. Successful entrepreneurs who are motivated in this fashion even if they succeed on the first try are combining both inspirational and experimental traits.
My guess is that few entrepreneurs succeed on their first try, and thus need to experiment or, to be blunt, to fail, often repeatedly, before achieving success. The secret economist I mentioned earlier who came to this conclusion much earlier than I did was Albert O. Hirschman, certainly one of the more unusual economists of the twentieth century, as should be evident from his profile in the following box.
Albert O. Hirschman: Unconventional Pioneer
The life of Albert O. Hirschman is one more suited to a spy novel than an economist, and fortunately it has been captured in a remarkable biography of the man and his ideas.21
Hirschman’s amazingly varied life began in Berlin where he was born. As the rise of Nazism became evident, he left Germany (with his family’s blessing) during his teenage years to study economics in France, England, and Italy, receiving his doctorate from the University of Trieste in 1938, just before the outbreak of World War II. An activist as well as intellectual, Hirschman punctuated his studies for a brief time by fighting against General Franco in the Spanish Civil War. After France surrendered to Germany, Hirschman risked his life again to help artists and intellectuals escape from Europe (and the Nazis) to the United States through the Pyrenees mountains in Spain and then to Portugal.
Hirschman immigrated to the United States during the early years of World War II, landing a fellowship at the University of California at Berkeley. He later joined the U.S. Army, which fortunately used his intellectual skills in the Office of Strategic Services (the predecessor to today’s Central Intelligence Agency). After the war, his academic career took him to a number of universities, in and outside the United States, which gave him a unique perspective on economics and societies.
Hirschman practiced an eclectic, word-based economics that is out of fashion today. He was a true public intellectual, drawing insights from multiple disciplines to look at the world the way it is—dynamic, messy, and complex—and not the way many economic theoreticians imagine it to be, simple, mechanical, and easily subject to mathematical description and analysis.
Though he was perhaps best known for his classic Exit, Voice, and Loyalty—which finds a role for each noun in any economy—Hirschman also makes some highly useful insights about entrepreneurs and their economic importance.22 In my paraphrasing of his description, the typical entrepreneur is not the swashbuckling risk-taker, but someone who embarks on a path that is so obvious to him or her that it would crazy to not do it, and thus is more accurately characterized as a “prudent” risk-taker. We know, by the way, from much modern research that for many, if not most, entrepreneurs, this is true—an insight that was drilled into me during many encounters I had with entrepreneurship researchers while I was at Kauffman.
But in Hirschman’s telling, here is where experimentation enters the picture. After launching his or her venture, the typical entrepreneur encounters any number of unexpected obstacles: Manufacturing the product is more difficult than expected, or consumers are not as enthusiastic about the initial version of the product or service as the entrepreneur anticipated. The successful entrepreneur is one who finds ways to overcome these obstacles, many times completely changing course again and again before hitting on what it is that consumers really want. The resulting success is often due more to accident than design, although the perseverance, and even madness (to outsiders), of the entrepreneur also play a critical role.
Hirschman describes this sequence of events in one of his essays, which Malcolm Gladwell recounts in his excellent review of Adelman’s biography of Hirschman.23 This essay describes Hirschman’s studies of large paper mills that were constructed in Pakistan with the intention of being supplied by bamboo forests from China. After those forests unexpectedly died, the mill’s operators improvised by building alternative supply chains, which ultimately made the mills more profitable than they would have been otherwise. Hirschman generalized from this experience by claiming what in my own experience rings true: Entrepreneurs typically start on a path they do not see as risky at all, then something happens that upsets their expectations and, being too far into their projects to turn back, they are forced to improvise. Those who get over this hump, or mountain, of unexpected trouble are the surviving entrepreneurial successes.
Hirschman’s account of entrepreneurship is similar in spirit to the much more recent description offered by Scott Adams, the creator of the Dilbert cartoon series. Adams has penned a thoughtful and highly readable account of both his life and how repeated failures ultimately help lead to success in almost any endeavor, including the founding and growth of a new firm.24
So much for economic descriptions and analysis about entrepreneurship and its role in economic advances, which are of interest to economists and to policy makers. But what have economists to say that can benefit entrepreneurs?
Here progress has been slower, in large part because producing this impact has not been high on many researchers’ agendas, and because reaching would-be and actual entrepreneurs is difficult. For one thing, whether entrepreneurs are inspirational or experimental, they are the type of people who do things for themselves, not the kind who consult books and articles, even though it is easier to do than ever before in the age of the Internet. For another, if they do read materials related to their chosen paths in life, entrepreneurs understandably are likely to be partial to things written by other entrepreneurs based on their own real-world experiences rather than by academics or journalists. Who can blame them?
There are limitations, however, with many of the entrepreneurial advice books one sees in bookstores or online. Too many are too general to be of much practical use, while others draw on personal anecdotes that do not often translate to the problems faced by particular entrepreneurs in their specific businesses. From an academic perspective, the advice books tend to be far from scientific because entrepreneurs (or consultants) turned authors are not trained to assemble large data sets and analyze them to draw general conclusions.
Even with these caveats, I can still safely recommend one popular book, coauthored by a successful entrepreneur, that I believe is the best practical guide for entrepreneurs ever written: Previously titled The Knack, its most current edition is entitled Street Smarts.25 The authors are Norm Brodsky, who has moonlighted for years as a columnist for Inc. (in my view, the best of the popular magazines for entrepreneurs) and his Inc. editor and coauthor Bo Burlingame, who has written widely about entrepreneurship for years.
As for the economists, they are slowly catching up, using the analytical and empirical tools of their trade to elicit insights from different data sets collected about entrepreneurs. One the best of this genre is the classic The Origin and Evolution of New Business by Amar Bhidé, now an economics professor at Tufts.26 Bhidé interviewed the founders or top executives of 100 companies no more than eight years old that had made the Inc. 500 in 1989, a list published each year by Inc. magazine of the fastest growing privately held companies over the previous four years. He supplemented this sample with another 200 case studies of successful entrepreneurs compiled by his students (when he taught at Harvard Business School), which he found to broadly corroborate the findings from his more formal sample.
<
br /> Bhidé’s purpose in writing his book was to uncover the keys to entrepreneurial success by identifying factors common among successful entrepreneurs. One of his main findings was that most founders had capital constraints, or limited money to start their ventures, which were generally financed by the founders’ own savings and borrowings (from credit cards and mortgages). Only 5 percent of the sample companies—and these were among the most successful private firms—were funded initially by more formal sources, such as venture capital firms. Subsequent research supported by the Kauffman Foundation over the years has confirmed this result.
Bhidé also confirmed that successful entrepreneurs mostly improvised or experimented as they went along. Few wrote or adhered to business plans, the focus of so much coursework and many entrepreneurial competitions since. The companies did not have seasoned professionals at the top (except for those financed by VCs, who often replace founders with more experienced executives when putting money into new companies), but generally were run by founders who were struggling and adapting as they went along. These findings emerged from both Bhidé’s formal sample and from his students’ case studies. For readers who want to learn more, I strongly urge reading Bhidé’s study, which has stood the test of time, and has been largely reinforced by more recent comprehensive studies of entrepreneurship, such as Noam Wasserman’s sure-to-be-classic The Founder’s Dilemmas, which I discuss shortly.
One drawback of the Bhidé study, however, is that it suffers from survivorship bias, because he looked only at many of the most successful companies. That was his objective, of course, but in doing so he implicitly made no attempt to establish a control group that inevitably would have had many unsuccessful or relatively less successful companies in his sample. Those less successful might also share some of the traits or keys to success Bhidé found among the most successful entrepreneurs, but we will never know.