There are three aspects of the Long Tail that have the effect of shifting demand down the tail, from hits to niches. The first is the availability of greater variety. If you offer people a choice of ten things, they will choose one of the ten. If you offer them a thousand things, demand will be less concentrated in the top ten.
The second is the lower “search costs” of finding what you want, which range from actual search to recommendations and other filters. Finally, there is sampling, from the ability to hear thirty seconds of a song for free to the ability to read a portion of a book online. This tends to lower the risk of purchasing, encouraging consumers to venture farther into the unknown.
There are several ways to try to quantify this with hard data. One is to compare a market that offers relatively limited variety with one that offers much more variety of the same sort of products. Another is to track a Long Tail aggregator/retailer over time, watching what happens as its inventory grows. Yet another would be to just look at the effect of lowered search costs online, making an apples-to-apples comparison with a similar offline inventory.
A 2005 study by a team at MIT lead by Erik Brynjolfsson, who did some of the early work on Amazon’s Long Tail inventory, looked at this effect at a women’s clothing retailer. The company has a catalog business and an online business, both of which offer the exact same inventory and prices. The difference is that online, it has search, easy browsing of both products and variations of those products, and ways to organize the offerings using “rank by” filters.
The result was that consumers—even those that shopped in both the catalog and online—tended to buy farther down the Tail online. The bottom 80 percent of products accounted for 15.7 percent of catalog sales, but 28.8 percent of online sales. Or to switch it around and see it from the top 20 percent perspective, the catalog exhibited an 84/20 rule, while the online site was closer to 71/20.
That’s the effect of lowered search costs for the same inventory. To measure the effect of different inventories—one much larger than the other—we worked to construct an apples-to-apples comparison between a retailer with limited shelf space and one with unlimited shelf space. In practice, that means comparing a bricks-and-mortar store with an online one selling or renting the same things. We decided to use entertainment examples because the online markets were mature enough to measure with confidence and the data was available. We looked at both music and DVDs.
Rather than pick a single bricks-and-mortar retailer, we used industry-wide data compiled by Nielsen divisions—SoundScan for music and DVDScan for movies. We compared that with online data from Rhapsody and Netflix, respectively.
(There are several corrections required to do these comparisons properly. In music we had to find a way to compare album sales offline with track sales online, and then from individual sales to streams under a subscription plan. In DVDs it was a matter of comparing sales and single-copy rental data offline with subscription rental online. Although the methodologies are beyond the scope of this book, they broadly revolve around using other data sets, such as pay-per-track online sales, to calibrate the curves and eliminate as many systematic biases as we could.)
After the corrections, the results were striking: The online demand curve is much flatter. The average niche music album title—those beyond the top 1,000—sold about twice as well online than offline. And the average niche DVD—again those beyond the top 1,000—was three times as popular online as it was offline.
Another way to look at this is to see how much less dominated the online market is by the top hits. Here’s the data for music. Offline, in bricks-and-mortar retailers, the top 1,000 albums make up nearly 80 percent of the total market. (Indeed, in a typical big-box retailer, which carries just a fraction of available CDs, the top 100 albums can account for more than 90 percent of the sales.) By contrast, online that same top 1,000 accounts for less than a third of the market. Seen another way, a full half of the online market is made up of albums beyond the top 5,000.
DOES THE LONG TAIL INCREASE DEMAND OR JUST SHIFT IT?
Does the Long Tail grow the pie or simply slice it differently? In other words, as the number of available products grows many-fold with the infinite shelf space of virtual retailers, does it encourage people to buy more stuff or just less popular stuff? In general, the answer depends on the sector: Some do seem to have huge opportunities for growth as their niches become widely available, and some do not.
Although human attention and spending power are finite, you can get more for your time and money. Some forms of entertainment, such as music, are “non-rivalrous” for attention, which is to say you can consume them while you’re doing something else. For instance, some explanations for the rise of average hours of TV watching in the seventies and eighties involved the idea that a generation had grown up used to television on in the background of their lives; as the novelty wore off, it went from a rivalrous to a non-rivalrous medium, and thus we consumed more of it.
Other media, such as text, may not be consumed faster, but they can be consumed more efficiently and with greater satisfaction through better preselection. Indeed, it’s quite extraordinary how much we’ve been able to increase our consumption bandwidth of information, scanning pages of Google search results and custom blog feeds. I may not read any more words than I once did, but they’re more likely to be meaningful to me, thanks to much better filters (better at suiting my own interests than, say, the editors of any newspaper) preselecting what I do read. So because the words are more relevant, my meaningful bandwidth has increased; I have, in a sense, compressed my reading attention.
But once you combine the scarcity of disposable income with the scarcity of time, some non-rivalrous media may become rivalrous. The reason people have the television on in the background is that it doesn’t cost them anything to do so. But if that were pay-per-view video, you can bet it would suddenly become the center of their attention. From a consumer perspective, this highlights the advantages of all-you-can eat subscription services, which offer risk-free exploration down the Tail. You’re likely to consume more if it doesn’t cost you more to do so.
So bottom line: Human attention is more expandable than money. The primary effect of the Long Tail is to shift our taste toward niches, but to the extent we’re more satisfied by what we’re finding, we may well consume more of it. We just won’t necessarily pay a lot more for the privilege.
SHOULD PRICES RISE OR FALL DOWN THE TAIL?
I’m often asked about the effect of the Long Tail on pricing. Should prices go down with demand as you travel down the Tail? Or should they rise, as more specific and narrowly focused goods appeal more strongly to their niche audiences?
The answer is that it depends on the product. One way to look at it is to distinguish between “want” markets and “need” markets, each of which has different implications for pricing.
Need markets are those in which customers know what they’re looking for and just can’t find it anywhere but, say, online. Take, for instance, a relatively hard-to-find nonfiction book on a topic of keen interest to you. When you find it, you’re probably going to be relatively price insensitive. You can see this effect writ large in the discounting policy at Amazon. The online bookseller discounts best-sellers by 30 to 40 percent, gradually reducing the discount until it’s around zero for the books with sales ranks in the hundred thousands.
By comparison, music and other forms of entertainment are typically “want” markets. For the right price, you can be encouraged to try something new, venturing down the Tail with diminished risk of wasting your money. Thus, many music labels have experimented with discount pricing for their older titles and more obscure new acts.
The ultimate manifestations of this would be dynamic variable pricing, where prices for music would automatically fall with popularity. That is, in fact, what Google does with its automatic auctions for keyword ads, and what eBay’s similar auctions do for everything else. The more demand there is, the higher
the price goes.
A really efficient variable pricing market would presumably lead to a more gradual sales decay, and a flatter demand curve overall. But for music, at least, the adoption of such a model runs up against the advantages of single-price simplicity (as in iTunes’ fixed-price $0.99 model) and the perils of dreaded “channel conflict” with CD retailers who cannot so easily change their prices. As the music industry gets more desperate it will probably grow more bold in its search for new business models. And then we’ll have better data with which to answer this question.
“MICROSTRUCTURE” IN THE LONG TAIL
One of the features of powerlaws is that they are “fractal,” which is to say that no matter how far you zoom in they still look like powerlaws. Mathematicians describe this as “self-similarity at multiple scales,” but what it means is that the Long Tail is made of many mini-tails, each of which is its own little world.
When you look closely at the data, you can see that the big powerlaw curve of, say, “music” is really just the superposition of all the little powerlaw curves formed by each musical genre. Music is made up of thousands of niche micromarkets, miniature ecosystems that, when smooshed together into an overall ranking, look like one Long Tail. But look closer and each has its own head and tail.
As an example, I’ve broken out the Long Tail of music on Rhapsody by genre (just the A’s), plotting the average track rank in terms of downloads for each genre on the plot below. I then broke out one—Afro-Cuban Jazz—even further, showing the curve of track popularity within that genre.
What you can see is that the genre averages themselves make up a Long Tail, and within each genre there is another Long Tail of individual tracks. And so it goes for the entire music universe, which appears to be one big popularity curve but is actually curves within curves within curves.
The same is true for other markets, from books to blogs. Peter Hirshberg, an executive at the blog search company Technorati, describes the emergence of “topical Long Tails” that the company has been tracking, showing the popularity powerlaw for such categories as cooking blogs and parenting blogs. “As when you apply a prism to white light, there is a spectrum of individual long tail communities in the blogosphere,” he says. Rankings are most meaningful within such communities, not across them.
Why does this matter? First, because it suggests that the filtering is often most effective at the genre level rather than across the entire market. And second, because it explains an apparent paradox of the Long Tail. The characteristic steep falloff shape of a popularity powerlaw comes from the effect of powerful word-of-mouth feedback loops that amplify consumer preference, making the reputation-rich even richer and the reputation-poor relatively poorer. Success breeds success. In network theory such positive feedback loops tend to create winner-take-all phenomena, which is another way of saying that they’re awesome hit-making machines.
Compounding matters, today’s filters make word-of-mouth even more powerful by measuring so much more of it from so many more people and for so many more products. Shouldn’t that then have the effect of making the powerlaw even steeper, increasing the gap between hits and niches rather than having a leveling effect?
In other words, why don’t network-effect recommendation systems, which are essential in driving demand down the Tail, actually do the opposite: drive content up the Tail, further amplifying hit/niche inequality? That’s what you’d expect with more powerful network effects, yet what we actually see in Long Tail markets is a flattened powerlaw, with less of a difference between hits and niches.
The explanation, it turns out, is that these filters and other recommendation systems actually work most strongly at the niche level, within a genre and subgenre. But between genres their effect is more muted. There are breakout hits that rise to the top of a genre and then go on to become mainstream hits, topping the overall charts. Yet they’re the exception. More common are titles that use their genre popularity to break into the middle of the overall charts, at which point they have to compete with many other hits from other genres and thus tend to not rise much farther.
Thus the most popular “ambient dub” artist at the very head of the ambient dub popularity curve can hugely outsell the others in that category, but that doesn’t mean that artist will snowball and tear up the charts to knock 50 Cent out of the top ten. The lesson from this microstructure analysis is that popularity exists at multiple scales, and ruling a clique doesn’t necessarily make you the homecoming queen.
THE LONG TAIL OF TIME
Why are some things less popular than others? So far we’ve been focusing mostly on the depth and breadth of certain items’ appeal—how mainstream or niche they are, or how high or low their quality. But there is another factor that influences popularity: age. Just as things of broad appeal tend to sell better than things of narrow appeal, new things tend to sell better than old things.
When you look at a basic demand curve, the reasons why some things sell less well than others are lost in the merged rankings. But popularity is actually multidimensional: Factors that determine an album’s rankings, for instance, can include not just the quality of the music but also its genre, its release date, the fame and/or nationality of the band, similarity to other artists, and so on. Yet it’s all blended into the single dimension of a best-seller list, which obscures all those factors in a mushed-together mélange.
If you think about it, today’s hit is tomorrow’s niche. Almost all products, even hits, see their sales decay over time. Twister was the number two movie of 1996, but its DVD version is now outsold two-to-one on Amazon by a 2005 History Channel documentary on the French Revolution.
Einstein described time as the fourth dimension of space; you can think of it equally as the fourth dimension of the Long Tail. Both hits and niches see their sales slow over time; hits may start higher, but they all end up down the Tail eventually. The research to quantify this conclusion is continuing, but conceptually the picture looks like the graph on the next page.
What’s particularly interesting about time and the Long Tail is that Google appears to be changing the rules of the game. For online media, like media anywhere, there is a tyranny of the new. Yesterday’s news is fishwrap, and once content falls off the front page of a Web site, its popularity plummets. But as sites find more and more of their traffic coming from Google, they’re seeing this rule break.
Google is not quite time-agnostic, but it does measure relevance mostly in terms of incoming links, not newness. So when you search for a term, you’re more likely to get the best page than the newest one. And because older pages have more time to attract incoming links, they sometimes have an advantage over the newer ones. The result is that the usual decay of popularity for blog posts and online news pages is now much more gradual than it was, thanks to the amount of traffic that comes via search. Google is in a sense serving as a time machine, and we’re just now being able to measure the effect this has on publishing, advertising, and attention.
THE TRAGICALLY NEGLECTED ECONOMICS OF ABUNDANCE
Broadly, the Long Tail is about abundance. Abundant shelf space, abundant distribution, abundant choice. How awkward, then, that one of the definitions of economics given by Wikipedia is:
eco-nom-ics: n The social science of choice under scarcity.
There are other definitions (“the allocation of scarce resources to satisfy unlimited wants,” and so on), but many share the same troublesome element: a focus on scarcity, especially on how to allocate scarce resources. In an age of abundance in the form of everything from Moore’s law (the observation that computer price/performance doubles every eighteen months) to its equivalents in storage and bandwidth, this is a problem.
It’s hard to overstate how fundamental to economics the notion is that you can’t have it all for free—the entire discipline is oriented around studying trade-offs and how they’re made. Adam Smith, for instance, created modern economics by considering the trade-off between time, or convenience
, and money. He discussed how a person could live near town, and pay more for rent of his home, or live farther away and pay less, “paying the difference out of his convenience.” And since then, economics has been all about how to divide finite pies.
That’s just the way it is. Neoclassical economics explicitly does not deal with abundant inputs. It doesn’t deny that oxygen is free when you’re trying to light a fire; it just doesn’t include that in its equations. It leaves that to other disciplines, such as chemistry.
But we are entering the era of effectively infinite shelf space. Two of the main scarcity functions of traditional economics—the marginal costs of manufacturing and distribution—are trending to zero in Long Tail markets of digital goods, where bits can be copied and transmitted at almost no cost at all. Surely economics has something to say about that?
Clearly abundance (also known as “plentitude”) is all around us, especially in technology. Moore’s Law is a classic example. What Carver Mead, the semiconductor pioneer and Caltech professor, recognized in 1970 when he encouraged his students to “waste transistors” was that transistors were becoming abundant, which is to say effectively free. The shift in thinking from making the most of scarce computing resources to “wasting” cycles by, say, drawing windows and icons on the screen led to the Mac and the personal computing revolution. To say nothing of the scandalous profligacy—a supercomputer used for fun!—of an Xbox 360.
We also have similar abundance laws working in storage and bandwidth and virtually everything else digital. Outside of technology, the green revolution brought abundance to much of agriculture (so now, to prop up prices, we pay farmers not to plant their crops). And what is the motive force behind China and India’s rise if not abundant labor, allowing them to, in a sense, waste people?
Even ideas can on some level be considered abundant, because they can propagate without limit due to their “non-rivalrous” nature. As Thomas Jefferson, the father of the U.S. patent system, put it, “He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me.”
The Long Tail Page 15