by Tim Hwang
The full economic impact of ad blocking worldwide is huge. Adobe estimated in 2015 that $21.8 billion in global ad revenue is lost each year to ad blockers.16 That’s a significant annual loss, more than all the revenue generated by Facebook that year.17 Relatively wealthier consumers also disproportionately adopt ad-blocking tools, reducing advertisers’ access to some of the most valuable audiences on the web.18 Generationally, the long-term trend is also worrisome. Ad blocking is significantly more prevalent in users eighteen to twenty-four than among older cohorts.19 Since younger users also pay less attention to ads, the advertising industry may be facing a demographic shift in behavior that will dramatically erode the viability of its existing model.
The seriousness of the threat posed by ad-blocking software to the online advertising sector can be measured by the vitriol with which representatives of the industry have responded. Randall Rothenberg, who leads the IAB, declared melodramatically that ad blocking is “robbery, plain and simple—an extortionist scheme that exploits consumer disaffection and risks distorting the economics of democratic capitalism.”20
It is worth recognizing that these arguments are about the aggregate trend of the online advertising economy. Some online advertising works, in certain contexts, for particular products. A close look at programmatic advertising does, in fact, reveal certain domains in which advertising remains highly effective. Famously, mesothelioma—the cancer caused by asbestos that has produced an active industry of class action lawsuits—remains one of the most expensive keywords to advertise against in search results, in part because people searching for this rare disease are in extreme need and are seeking treatment.21
But such domains are the exception, rather than the norm. Undermined by visibility problems, indifference, and ad blocking, online advertising overall is increasingly subprime. It fails to truly capture attention or influence consumer behavior. And it gets worse. As the quality of real attention on offer declines, fraudsters continuously work to inflate the value of the attention inventory being sold. This poses a problem that, unchecked, could destabilize the economy of the web.
The Problem of Ad Fraud
The sheer size of the online advertising market invites attempts to fraudulently extract money from it. Exploitation is rampant in the online advertising ecosystem. Perversely, these activities actually boost the perceived value of advertising inventory—at least in the short term.
The basic idea of ad fraud is simple: buyers think they are purchasing attention and delivering their message to promising consumers, but instead they are given something worthless. There are numerous ways that this can happen.
Click fraud is a widespread practice that uses automated scripts or armies of paid humans in “click farms” to deliver click-throughs on an ad. The result is that the advertising captures no real attention for the marketer. It is shown either to a human who was hired to click on the ad or to no one at all.
The scale of this problem is enormous. A study conducted by Adobe in 2018 concluded that about 28 percent of website traffic showed “non-human signals,” indicating that it originated in automated scripts or in click farms.22 One study predicted that the advertising industry would lose $19 billion to click fraud in 2018—a loss of about $51 million per day.23 Some place this loss even higher. One estimate claims that $1 of every $3 spent on digital advertising is lost to click fraud.24
Display advertising, which still constitutes one of the largest segments of the digital advertising market, exhibits these problems in a particularly striking way. In 2017, Forrester Research concluded that the previous year, as much as 56 percent of all ad dollars spent on display advertising were lost to fraudulent or otherwise unviewable inventory. This accounted for $7.4 billion of waste, which was estimated to grow to $10.9 billion by 2021.25
There is good reason to believe that fraud will continue to destabilize the industry, even as advertisers find new ways of delivering marketing messages through the web. Video, one of the fastest-growing segments of the programmatic advertising market, is similarly plagued by fraudulent traffic. Fraud accounts for 22 percent of video spend, and 20 percent of nonmobile video traffic is driven by bots. If anything, the problem is worse in video than it is in display. One study concluded that fraud was twice as common in video traffic as in display traffic.26 Mobile, another quickly growing segment, is likewise vulnerable to fraud. Eighty-seven percent of all mobile devices on offer in the programmatic advertising markets in the United States in fall 2016 were fraudulent, meaning that they were either not real phones at all, or they were phones running automated scripts, unseen by any actual members of the public.27
Domain spoofing is another popular scam, in which inventory on the programmatic marketplace is made to look like space on a high-value website when it, in fact, is not. Unwitting ad buyers pay top dollar for these opportunities, not realizing that their ads appear somewhere else entirely. In 2017, the Financial Times discovered that fraudsters were pretending to sell Financial Times ad inventory on ten different ad exchanges. In fifteen ad exchanges, they also found offers to sell video advertising on the Financial Times site, a type of inventory the newspaper didn’t even offer its legitimate advertisers. The fake inventory was selling for approximately $1.3 million per month, which the advertising operations director for the publication called “jaw-dropping.”28
Ad fraud flourishes because it is highly lucrative. In 2018, The New York Times profiled one “entrepreneur” in the space, Martin Vassilev, who in eighteen months went “from being on welfare and living with his father in Canada to buying a white BMW 328i and a house of his own.” Small operators delivering fake YouTube views with software can earn more than $30,000 per month.29
Ad fraud also supports the development of more sophisticated criminal enterprises that can target established publishers and even threaten the Google-Facebook duopoly. These advanced operators leverage malicious code to direct the traffic of unknowing consumers to support these ad fraud schemes. In 2018, a BuzzFeed story exposed a $10 million scheme that used more than 125 Android apps and website ads to generate fake traffic.30 Methbot, a “professionalized” ad fraud scheme uncovered in 2016, generated between $3 million and $5 million per day at its height by impersonating inventory on nearly six thousand publications including The Economist, ESPN, Fortune, and Facebook.31
The vast cumulative impact of fraud in the programmatic advertising ecosystem is worth taking a moment to put in everyday terms. Imagine a supermarket with the level of inventory fraud that exists in the programmatic marketplace. In some aisles, one out of every five products on the shelves is a fake: you’d return home to find that these boxes contain nothing.32 Even when you stick to familiar brands that you recognize and trust—a six-pack of Coca-Cola or a box of Wheaties—your purchases turn out to be imitations filled with sawdust. You’d probably stop going to this supermarket, even if you did occasionally manage to get some of the groceries you wanted.
The perverse effect of ad fraud in the short and medium term is that it actually inflates the purported value of the global marketplace for digital advertising. Fake traffic driven by click farms and botnets makes ads look better and more effective than they actually are. Practices like domain spoofing expand the apparent supply of opportunities to get an advertising message to high-value audiences. These fraudulent practices juice demand for online advertising and pump up the overall size of the marketplace.
All the while, ad fraud erodes the true value of the underlying attention inventory. As fraud spreads, a larger and larger proportion of the dollars spent on advertising fails to capture any real or worthwhile attention. Fraudulent practices also erode trust, because any real attention captured through these channels becomes more and more suspect. How is an advertiser to know that someone actually saw the ad that it paid to place? An expanding gap appears between the market price of advertising and the true value of the asset. Pervasive fraud contributes to online advertising’s becoming increasingly subprime,
capturing a declining proportion of real attention and an increasing proportion of fake attention.
Defusing the Crisis?
The problems of declining attention, ad blocking, and fraud that we have discussed are well-known within the programmatic advertising industry. They all represent structural threats to the continued growth and vitality of the online advertising markets and have been treated as such by the sector. This has led to a variety of attempts to intervene and fix the ecosystem, though it is far from clear whether these efforts have been effective in changing the overall picture.
Companies responsible for purchasing large quantities of advertising increasingly scrutinize their marketing agencies and demand greater accountability.33 Buyers have pressured big platforms like Facebook and Google to allow more extensive third-party involvement in assessing the veracity of ad performance metrics.34
Technological solutions have also been developed. Introduced in 2017, ads.txt is one standard endorsed by the IAB to combat the problems of domain spoofing. The IAB encourages publishers to host a text file on their web servers containing an authoritative list of the sellers with permission to sell their inventory.35 At least in theory, this would allow buyers to assess whether a vendor on a programmatic exchange is engaging in domain spoofing. Driven by enthusiastic adoption from Google, ads.txt use has risen sharply; among the top thousand sites selling programmatic ads, 57 percent now host these reference files.36
The fight to save the structure of online advertising also means fighting ad-blocking software. The advertising industry has worked to erode the effectiveness of ad blockers.37 This is sometimes facilitated by the ad blockers themselves, who monetize their software by charging a premium to advertisers in exchange for access to people who block ads. Adblock Plus—one of the most popular companies operating in the space—launched its Acceptable Ads Platform in 2016, effectively creating a new ad network through which advertisers meeting certain criteria could access the ad blocker’s user base.38
The other approach taken by the advertising industry has been to grow past the problem. Rather than take on the daunting task of repairing existing marketplaces plagued by fraud and subprime attention, the strategy is to expand programmatic advertising into new media where it might capture higher-quality attention. The expansion into video, mobile platforms, and games reflects a strategy to create an alternative that might allow the industry to dump the existing workhorse of display advertising.
Despite these multifaceted efforts, the fundamentals have not changed. Ad fraud continues to be a cat-and-mouse battle. When old vulnerabilities are patched up, fraudsters find new ways to extract money illegitimately from the marketplace. The net effect is that fraud in these channels is not only persistent, but on the rise. One industry analysis suggests that ad fraud will more than double in the coming years, eventually reaching $44 billion by 2022, an amount equivalent to about 9 percent of the overall digital ad spend.39
Tactics like domain spoofing remain effective even with the advent of tools like ads.txt. One 2018 report showed that while ads.txt did lower the rate of fraud, the overall rate remained high.40 Even supposedly “safe” domains still saw fraud in excess of 13 percent.41 Fraudsters are nimble in this new environment; nothing about ads.txt prevents the introduction of fraudulent entities to individual websites’ “approved” lists. Publishers have been threatened and tricked into adding suspicious names to their lists.42 A scheme uncovered by researchers in 2018 specifically targeted vulnerabilities of the ads.txt program and could have cost the industry $70 million to $80 million had it gone unchecked.43
Nor has continued innovation to grow past the problem proven to be a particularly promising solution. Advertising in online video or through mobile platforms remains a relatively small sliver of the digital ad market. Neither is currently positioned to generate enough ad revenue to supplant search or display advertising. At the same time, many of these new channels suffer from the same problems of fraud, sometimes at even greater rates than before. Mobile ad fraud is “surging”; by some estimates it doubled in 2017.44
And always lurking in the background is the specter of a public uninterested in online advertising. The adoption of ad-blocking programs continues to rise globally, with particularly rapid adoption on mobile devices.45 This accompanies evidence that younger cohorts of internet users pay less and less attention to ads on new platforms. Snapchat—which has a disproportionately young user base—has been plagued by the fact that its users reliably “skip commercial interruptions within less than a second.”46 One 2017 survey of eighteen- to thirty-five-year-olds shows that fully 59 percent watch online video ads only until they can skip them, and another 11 percent block ads entirely.47
Past performance is, of course, no indication of future returns. The advertising industry might identify new ways of delivering ads online that can scale and capture attention at the same levels the original banner advertisements did in the 1990s. It might triumph in the long term over ad blockers and the scams that currently erode value in the advertising marketplace. Achieving these goals would require massive coordination between players in the programmatic marketplace and more than a few painful compromises.
There are real reasons to doubt that the market’s structural weaknesses will be corrected. Financial crises, as we have seen, can emerge even when players in the market know the risks of a growing bubble or the underlying weakness of a widely hyped asset. People who know the dangers of an overheated market and who have the ability to change it are seldom able to stem the tide of a crisis—in the rare cases when they try at all. The online attention markets of the web are exhibiting similar qualities, raising the question: What allows a bubble to keep growing, and what might cause it to pop?
5
Inflating the Bubble
Opacity and subprime attention create the necessary conditions for a market bubble in online advertising. This chapter examines how this dysfunctional market might grow overheated and then suddenly implode.
Granted, the advertising industry may resolve its many structural issues before a market crisis comes to pass. Optimists point to recent attempts to address fraud and wean people from ad blocking. They argue that it is still early for emerging channels like video and that many problems of inattention will be ironed out over time. These hypothetical changes to the industry might slowly let the air out of a bubble, preventing a sudden, damaging crash. However, these efforts would need to be effective in a way that they haven’t been to date.
Just because a problem is known doesn’t mean that it will be solved in time. The history of financial crises shows numerous situations in which buyers, sellers, and the companies running the markets failed to take sufficient action to avert a market bubble. That history has lessons here.
Two complementary factors are important. On the demand side, there needs to be a flow of cash into a marketplace that inflates a bubble in spite of the marketplace’s structural weaknesses. On the supply side, various players need to benefit from this flow of money. It becomes in their financial interest to disregard warning signs, limit structural changes, and keep the market growing even as alarm bells ring. Perverse incentives of this kind were a central cause of the 2008 subprime mortgage crisis and are present in the modern online advertising ecosystem today.
The Cash Bubble
Why aren’t advertisers more wary of all the structural issues that we’ve discussed? The murkiness of the programmatic advertising market and the challenges of subprime attention are well-known, but advertisers continue, quarter after quarter, to pump money into the digital ad economy.
One might argue that this continued investment means that these problems don’t pose a serious threat to the integrity of the overall market—that people are, in effect, signaling their confidence with their dollars. Parallels to past crises in financial markets offer an alternate hypothesis: advertising budgets are pouring into this market bubble because they have nowhere else to go. Let’s return to the subpr
ime mortgage crisis for an illustrative example.
One prevailing thesis about the origins of the 2008 crisis is known among economists as the “savings glut” hypothesis. This theory observes that fiscal instability in the 1990s encouraged developing countries around the turn of the twenty-first century to aggressively shift their money into safe assets. This process triggered a wave of purchases of U.S. Treasuries—debts of the U.S. government—by countries like China. U.S. Treasuries are widely considered the safest investments available, given that they are backed by the credit of the United States.
The scale of this shift toward safe investments was massive. Between 1996 and 2003, developing countries as a group shifted about $293 billion in this way, changing from net importers of capital (that is, bringing money from other countries into their country) to massive net exporters of capital (sending money to other countries).1 Much of that money ended up in the United States. The U.S. Financial Crisis Inquiry Commission, which was responsible for investigating the origins of the 2008 crisis, would later note that “from 2000 to 2006, U.S. Treasury debt held by foreign official public entities rose from $0.6 trillion to $1.43 trillion; as a percentage of U.S. debt held by the public, these holdings increased from 18.2% to 28.8%.”2
The savings glut hypothesis suggests that this massive shift in the movement of money created the circumstances for a broader crisis. Demand for these safe assets was so ravenous that the price of U.S. Treasuries rose, driving down their overall return. This prompted foreign investors and financial institutions to seek out new alternatives—assets that were just as safe but with higher returns. Securitized mortgages, with their reputation for stable growth over the long term, seemed like an ideal solution.