The Long Tail

by Chris Anderson

  More than a decade ago George Gilder, the apostle of abundance, offered a good way to think about all this:

  For most of human history, most people have believed that economics is essentially a zero-sum game—that scarcity will ultimately prevail over abundance. Pastor Malthus was the famous exponent of the view that populations increase geometrically while agricultural output rises arithmetically. In the Malthusian view, food scarcity eventually chokes off growth. Karl Marx saw all economics ultimately reducing to a class struggle over scarce “means of production.”

  The economists’ focus on scarcity stems from the fact that shortages are measurable and end in zero. They constrain an economic model to produce a clearly calculable result, an identifiable choke point in the industrial circuitry. Abundances are incalculable and have no obvious cap. When they are ubiquitous, like air or water, they are invisible—“externalities.” Yet abundance is the driving force in all economic growth and change.

  So how to reconcile this with neoclassical economics? Gilder recommends embracing waste.

  In every industrial revolution, some key factor of production is drastically reduced in cost. Relative to the previous cost to achieve that function, the new factor is virtually free. Physical force in the industrial revolution became virtually free compared to its expense when it derived from animal muscle power and human muscle power. Suddenly you could do things you could not afford to do before. You could make a factory work 24 hours a day churning out products in a way that was just incomprehensible before the industrial era. It really did mean that physical force became virtually free in a sense. The whole economy had to reorganize itself to exploit this physical force. You had to “waste” the power of the steam engine and its derivatives in order to prevail, whether in war or in peace.

  That suggests a way to put this in an economic context. If the abundant resources are just one factor in a system otherwise constrained by scarcity, they may not challenge the economic orthodoxy. They are then like learning curves and minimized transaction costs—drivers of production efficiency that serve to lower prices and increase productivity but do not invalidate the laws of economics.

  And, indeed, the abundance of the Long Tail, for all its power, is surrounded by such constraints. Although there may be near infinite selection of all media, there is still a scarcity of human attention and hours in the day. Our disposable income is limited. On some level, it’s still a fixed-pie game. Offer a couch potato a million TV shows and he or she may end up watching no more television than before, just different television, better suited to that individual.

  Finally, it’s worth noting that economics, for all its charms, doesn’t have the answer to everything. Many phenomena are simply left to other disciplines, from psychology to physics, or left without an academic theory at all. Abundance, like growth itself, is a force that is changing our world in ways that we experience every day, whether we have an equation to describe it or not.

  9

  THE SHORT HEAD

  THE WORLD THE SHELF CREATED, FOR BETTER OR WORSE

  Hits, like it or not, are here to stay. So are retail stores with limited shelf space and broadcast networks, lowest-common-denominator fare and all. For all the growth in e-commerce, online shopping is still less than 10 percent of American retail, having just passed catalog shopping. Even the biggest boosters of online shopping don’t expect that they’ll pass a quarter of consumer spending for decades.

  It’s not just the instant-gratification convenience and tactile advantages of bricks and mortar. We’re also a gregarious species, and sometimes we like to do things together with other people. There’s comfort in numbers, and shared experiences bring us closer.

  That’s why the unequal shape of powerlaws is unavoidable. Long Tail markets tend to be a bit flatter than traditional markets, but they still have their share of blockbusters. For each way that we differ from one another, there are more ways that we’re alike. This is not only inevitable, but it’s actually essential in helping kickstart recommendations and other filters that make the rest of the online market work.

  In this chapter we’ll return to the left side of the powerlaw, the land of the A list. We’ll look at both the virtues of shelves and their costs, and likewise for broadcast technologies and Hollywood’s hit-making machine. Let’s start with their advantages.

  Hits may not dominate society and commerce as much as they did over the past century, but they still have unmatched impact. And part of that is their ability to serve as a source of common culture around which more narrowly targeted markets can form.

  Successful Long Tail aggregators need to have both hits and niches. They need to span the full range of variety, from the broadest appeal to the narrowest, to be able to make the connections that can illuminate a path down the Long Tail that makes sense for everyone.

  Consumers want one-stop shopping. They want to have some confidence that what they’re looking for is in one specific place. Stores that give consumers the most confidence that everything they want is there are going to be the ones that succeed. This notion of ultimate selection, of knowing that the filters are selecting the best from a choice of everything (or at least everything in that domain) is why good Long Tail aggregators are so compelling.

  If you just have the products at the Head, you find that very quickly your customers want more and you can’t offer it. If you just have the products at the Tail, you find that customers have no idea where to start. They’re unable to get traction in the marketplace because everything you’re offering is unfamiliar to them. The importance of offering the stuff at both the Head and the Tail is that you can start in the world that customers already know: familiar products that tap into and define a space.

  A good example of why this is so necessary is the story of MP3.com, one of the early online music services. In 1997, an entrepreneur named Michael Robertson started what looked like a classic Long Tail business. It let anyone upload music files that would be available to all. The idea was that the service would bypass the record labels, allowing artists to connect directly to listeners. MP3.com would make its money in fees paid by bands to have their music promoted on the site. The tyranny of the labels would be broken, and a thousand flowers would bloom.

  But although MP3.com grew quickly and soon had hundreds of thousands of tracks, struggling bands did not, as a rule, find big new audiences, and independent music was not transformed. Indeed, MP3.com got a reputation for being exactly what it was: an undifferentiated mass of mostly bad music that deserved its obscurity.

  The problem with MP3.com was that it was only Long Tail. For most of its life it didn’t have license agreements with the labels to offer mainstream fare or much popular commercial music at all. Therefore, there was no familiar point of entry for consumers, no known quantity from which further exploring could begin. (In the search for a viable business model, it later offered a service that allowed users to upload commercial CDs they owned, which brought on massive record industry lawsuits that eventually shut down the company.)

  The reason MP3.com’s model didn’t succeed and the iTunes model—which is less oriented toward independent musicians—did is that iTunes began by making deals with major record labels, which gave it a critical mass of mainstream music. Then it added more and more niche content, as “rights aggregators” shipped it hard drives full of hundreds of thousands of independent musicians. Thus, iTunes customers were able to dive into an already working market where the categories were defined by known commercial acts, which served as a natural leaping-off point for the discovery of niche music.

  (As an aside, it’s worth asking why MySpace, which has a free independent music model that is very reminiscent of MP3.com, is such a success. The answer at this point appears to be that it is a very effective combination of community and content. The strong social ties between the tens of millions of fans there help guide them to obscure music that they otherwise wouldn’t find, while the content gives them a reason to keep
visiting. This helps the site avoid the burnout phenomenon that has sunk previous social networking services that were mostly about connection for connection’s sake.)

  THE URBAN TAIL

  Another sort of “hit” is major cities. If you chart population clusters around the globe, you’ll get a powerlaw. A small number of places, from Shanghai to Paris, have huge populations, while many more places have smaller populations. As Richard Florida, in his book The Rise of the Creative Class, puts it, “the world is spiky”:

  People cluster not simply because they like to be around one another or they prefer cosmopolitan centers with lots of amenities, though both those things count. They and their companies also cluster because of the powerful productivity advantages, economies of scale, and knowledge spillovers such density brings. Ideas flow more freely, are honed more sharply, and can be put into practice more quickly when large numbers of innovators, implementers, and financial backers are in constant contact with one another, both in and out of the office.

  These population spikes—the great cities of the world—exist because the cultural and economic advantages of being around lots of other people more than compensate for the costs of urban living. One of those advantages, ironically enough, is massive variety in every possible niche.

  Places like New York City, London, Paris, and Tokyo offer practically everything. Want international food? It’s all there—from Eritrean and Bengali to Mongolian hot pots. There is entertainment of every possible variety, services to cater to every need, and if you know which side street or hole in the wall to explore, a bounty of products to rival even Amazon.

  Why? Because cities have such a dense population that the usually widely distributed demand becomes concentrated. In a sense, you can think of cities as the Long Tail of urban space in the same way the Internet is the Long Tail of idea space or cultural space.

  As the writer Steven Johnson puts it:

  A store selling nothing but buttons most likely won’t be able to find a market in a town of 50,000 people, but in New York City, there’s an entire button-store district. Subcultures thrive in big cities for this reason as well: if you have idiosyncratic tastes, you’re much more likely to find someone who shares those tastes in a city of 9 million people.

  The urban theorist Jane Jacobs observed many years ago that huge cites create environments where small niches can flourish. She wrote:

  Towns and suburbs…are natural homes for huge supermarkets and for little else in the way of groceries, for standard movie houses or drive-ins and for little else in the way of theater. There are simply not enough people to support further variety, although there may be people (too few of them) who would draw upon it were it there.

  Cities, however, are the natural home of supermarkets and standard movie houses plus delicatessens, Viennese bakeries, foreign groceries, art movies, and so on, all of which can be found coexisting, the standard with the strange, the large with the small. Wherever lively and popular parts of cities are found, the small much outnumber the large.

  IN DEFENSE OF SHELVES

  Before we bury the shelf, let us first praise it. Today’s retail display rack is the human interface to a highly evolved supply chain designed to make the most of time and space. Standing as much as seven feet high and four feet wide and extending up to two feet deep, the average supermarket shelf module has the cubic capacity of a minivan.

  Stacked with hundreds of packaged goods designed to fit perfectly in industry-standard racks, that shelf has become the modern-day symbol of abundance. Today the average supermarket carries more than 30,000 different items, all ideally arranged and displayed in rows of shelves for maximum sales at minimum cost. It is both a miracle of efficient storage and a fine-tuned selling machine.

  The shelf reflects the absolute state of the art in retail science. The products on today’s supermarket shelves are packaged and arranged according to stocking algorithms and the peaks of elastic demand curves. The optimal inventory distribution is recalculated each day in retail chain headquarters and tuned in real time on the basis of checkout data.

  These shelf-stocking models are designed to press every button we’ve got: satisfying existing demand, stimulating new demand, and extracting the highest possible sales from the smallest space. Every dimension of the supermarket shelf has been studied, focus-grouped, and observed by retail anthropologists via hidden cameras and radiofrequency ID tags. The retail shelf is the frontline of an enterprise that accounts for nearly 60 percent of the American economy, and the research industry devoted to understanding it befits its importance.

  We know the precise value gradient of the vertical dimension in a rack of shelves, from top to bottom. We also know the exact dollar value of the golden shelf just below eye level in each product category and type of retail (for instance, in supermarkets that magic place in the middle has more than five times the selling power of the bottom shelf). As a result, stores determine exactly how high a “slotting fee” they can charge manufacturers to place their products in this purchasing sweet spot, increasing both sales for the maker and margins for the seller.

  Meanwhile, the horizontal dimension is a study in optimizing brand exposure. We now know exactly how wide to stack a company’s products to capture a shopper’s scanning eye without going too wide and wasting scarce shelf frontage. Thanks to bar codes and point-of-sale integration with stock replenishment software, we also know how to keep shelves filled with the right stuff all the time.

  In short, thanks to decades of research by the best minds in supermarketology, we have learned how to make the most of each square inch of retail space. When one considers how far we’ve come, with the explosion of abundance and variety and the price-lowering effect of global supply chains, it is hard to quibble with the shelf. It is the very embodiment of capitalism evolved.

  RENT BY THE HALF INCH

  Yet the shelf is so wasteful in so many ways. Let’s start with the obvious. The monthly rent on that two-by-four-foot slab is outrageous. True, you can stack six square feet of shelf space on top of one square foot of floor space, but retail rules of thumb dictate that for every square foot of floor space used for shelves, you need another two to three square feet of aisle, checkout, and common space. Depending on the type of store, backroom storage and administrative space can add another 25 percent to the floor space required. In early 2005, mall retail space in major U.S. markets was renting for an average of nearly $40 per square foot; this can put the net space cost of each square foot of shelving at between $26 and $33 a month.

  Then there are the other overheads of bricks-and-mortar retail: sales staff, inventory depreciation, power and other utilities, shoplifting and other “leakage” issues, returns, insurance, and marketing costs. Combined, these types of overhead can nearly equal the space costs, bringing the total rent on that twelve-inch-by-twelve-inch square of shelving to at least $50 a month. With an average retail markup of 40 percent, this means that the average square foot of mall shelf space must account for between $100 and $150 in sales a month—and that’s to simply pay its way.

  Since every slot on that shelf is precious, only the most promising products—those with a certain expected popularity or profit margin—can be allowed in. It’s a brutal test, and the vast majority of products don’t make it. Supermarkets consider 15,000 new products a year. Out of the few that actually make it to the shelf, an estimated 70 to 80 percent don’t survive for long there, according to Consumers Union. Today, the average cost of carrying a single DVD in a movie rental store is $22 a year. Only the most popular titles rent often enough to make that back (there’s a reason why they call it “Blockbuster”).

  If that weren’t bad enough, the hidden costs of selling products on shelves can actually be even higher than the direct costs. These are largely the opportunity costs of products not found and latent demand not realized because of the physical constraints of shelves. The Google era has opened our eyes to the lucrative virtues of findability. We type in w
hat we want (misspelled or not) and, more often than not, it pops right up. We are now spoiled with useful recommendations (lessons learned by those who came before us) that introduce us to things we never would have thought of or found on our own.

  Yet none of this carries over to the local Safeway, where products are shoehorned into crude taxonomies (“canned goods”), the patterns of other shoppers are known only to management, and the only search engine available is a stock clerk who works for minimum wage. This is not really Safeway’s fault, or the fault of any other bricks-and-mortar retailer. Those retailers are simply fated to live in the inflexible world of racks and aisles, where products must obey the uncompromising physics of atoms, not bits.

  One of those unfortunate rules of corporeal matter is that it cannot transcend time and space. Obviously, a physical item can be in only one place at any given time. For instance, a can of tuna cannot exist simultaneously in multiple categories, even though the interests and browsing paths of each shopper might suggest many: “fish,” “canned food,” “sandwich makings,” “low-fat,” “on sale,” “best-selling,” “back-to-school,” “under $2,” and so on.

  A physical store cannot be reconfigured on the fly to cater to each customer based on his or her particular interests. Bottles of wine cannot be magically rearranged to suit the results of a search. They cannot be popped onto the next shelf to optimize the probability that people like you who bought aged Gouda and black olives might also like this Pinot. Atoms are stubborn this way.