The Economics of Artificial Intelligence
Page 74
tive algorithms often assume there is a hidden truth to learn, which could be
the consumer’s gender, income, location, sexual orientation, political prefer-
ence, or willingness to pay. However, sometimes the to-be- learned “truth”
evolves and is subject to external infl uence. In that sense, the algorithm may
intend to discover the truth but end up defi ning the truth. This could be harmful, as algorithm developers may use the algorithms to serve their own
interest, and their interests—say earning profi ts, seeking political power,
or leading cultural change—could confl ict with the interest of consumers.
The danger of misleading algorithms is already seen in the controversy
about how Russia- sponsored posts got disseminated in social media during
the 2016 US presidential election. In the congressional hearings held on
October 31 and November 1, 2017, lawmakers expressed the concern that
13. https:// newsroom.fb .com/ news/ 2017/ 09/ information- operations-update/ , accessed on October 19, 2017.
14. https:// www .nytimes .com/ 2017/ 10/ 30/ technology/ facebook- google- russia .html, accessed on December 18, 2017.
15. https:// www .nytimes .com/ 2017/ 09/ 07/ us/ politics/ russia- facebook- twitter- election .html, accessed on October 19, 2017. http:// money.cnn .com/ 2017/ 09/ 28/ media/ blacktivist- russia
- facebook- twitter/ index .html, accessed on October 19, 2017.
Artifi cial Intelligence and Consumer Privacy 447
the business model of Facebook, Twitter, and Google, which depends on
advertising revenue from a large user base, may hamper their willingness to
identify or restrict misinformation from problematic users.16 Because social
media users are more likely to consume information that platform algo-
rithms push to them, they may end up consuming information that hurts
them in the future.17
The same confl ict of interest has sparked concerns in price discrimina-
tion. This argument is that if AI enables a fi rm to predict a consumer’s
willingness to pay, it could use that information to squeeze out every penny
in consumer surplus. This argument is plausible in theory, but needs to be
evaluated with at least three considerations: fi rst, if more than one fi rm
can use AI to discover the same consumer willingness to pay, competi-
tion among them will ease the concern of perfect price discrimination;
second, the economics literature has long demonstrated the ambiguous
welfare eff ect of price discrimination. As long as price discrimination is
imperfect (i.e., fi rms cannot charge every consumer’s willingness to pay),
some consumers may benefi t from the practice (via lower price) while other
consumers suff er. From a social planner’s point of view, whether to encour-
age or punish AI- enabled price discrimination depends on the weights it
assigns to diff erent parts of society. Third, in the long run, AI may reduce
the operational costs within the fi rm (e.g., via a more cost- eff ective inven-
tory management system) and foster product innovations that better fi t
consumer demand. These changes could be benefi cial to both the fi rm and
its consumers.
A somewhat opposite concern is that AI and other predictive technology
are not 100 percent accurate in their intended use. It may not introduce
much ineffi
ciency or wasteful eff ort if Netfl ix cannot precisely predict the
next movie I want to watch, but it could be much more consequential if the
US National Security Agency (NSA) fl ags me as a future terrorist based on
some AI algorithm. As Solove (2013) has argued, it is almost impossible for
someone to prove that they will not be a terrorist in the future. But at the
same time, they may be barred from air travel, have personal conversation
with friends monitored, and be restricted from work, trade, and leisure ac-
tivities. If this AI algorithm applies to a large population, it could do a lot
of harm even if the probability of error is close to zero.
To summarize, there is a real risk in privacy and data security. The magni-
16. The full video and transcript of these hearings are available at c- span .org (https:// www
.c- span .org/ video/ ?436454– 1/ facebook- google- twitter- executives- testify- russia- election- ads, and https:// www .c- span .org/ video/ ?436360– 1/ facebook- google- twitter- executives- testify
- russias- infl uence- 2016-election&live).
17. Note that a predicative algorithm is not necessarily more biased than human judgment.
For example, Hoff man, Kahn, and Li (2018) study job- testing technologies in fi fteen fi rms.
They fi nd that hires made by managers against test recommendations are worse on average.
This suggests that managers often overrule test recommendations because they are biased or mistaken.
448 Ginger Zhe Jin
tude of the risk, and its potential harm to consumers, will likely depend on
AI and other data technologies.
18.3 How Does the US Market Cope with
the Risk in Privacy and Data Security?
Before we jump into a regulatory conclusion, we must ask how the market
copes with the risk in privacy and data security. Unfortunately, the short
answer is that we do not know much. Below I describe what we know on the
demand and supply sides, along with a summary of existing public policies
in the United States. Admittedly, the literature cited below is more about
privacy and data security than about AI. This is not surprising, as AI has
just started to fi nd its way into e-commerce, social media, national security,
and the internet of things. However, given the ongoing risk and the potential
interaction of AI and that risk, it is important to keep in mind the big picture.
18.3.1 Consumer
Attitude
On the demand side, consumer attitude is heterogeneous, evolving, and
sometimes self- confl icting.
When surveyed, consumers often express serious concerns about privacy,
although self- reported value of privacy covers a wild range (see the sum-
mary in Acquisti, Taylor, and Wagman [2016]). However in real transactions,
many consumers are willing to give away personal data in exchange for a
small discount, free services, or a small incentive such as a pizza (Athey,
Catalini, and Tucker 2017). This confl ict, which some referred to as a “pri-
vacy paradox,” suggests that we have yet to comprehend the link between
consumer attitude and consumer behavior. Furthermore, researchers have
found that privacy preference varies by age (Goldfarb and Tucker 2012),
by time (Stutzman, Gross, and Acquisti 2012), and by context (Acquisti,
Brandimarte, and Loewenstein 2015). Although old data are shown to add
little value to search results (Chiou and Tucker 2014), biometric data such
as fi ngerprint, facial profi les, and genetic profi les can be much longer lasting
(Miller and Tucker, forthcoming). Hence, consumers may have a diff erent
preference on biometric data than on the data that gets obsolete fast. These
heterogeneities make it even harder to paint a complete picture of consumer
attitude and consumer behavior about privacy.
A similar puzzle exists for attitudes toward data security. A recent sur-
vey by the Pew Research Center suggests that many people are concerned
high- profi le data breaches (Pew Research Center 2016). However, accord-
ing to Ablon et al. (2016), only 11 percent stopped dealing with the aff ected
company and 77 percent were highly satisfi ed with the company’s postbreach
response.
It is hard to tell why consumers are willing to give away data in real trans-
Artifi cial Intelligence and Consumer Privacy 449
actions. One possibility is that consumers have a large or even hyperbolical
discount for the future, which motivates them to value the immediate gains
from the focal transaction more than the potential risk of data misuse in the
distant future. Other behavioral factors can be at play as well. Small incen-
tives, small navigation costs, and irrelevant but privacy- reassuring infor-
mation can all persuade people to relinquish personal data, according to a
recent fi eld experiment (Athey, Catalini, and Tucker 2017).
It is also possible that news coverage—on data breaches and privacy
problems—raises consumer concern about the overall risk, but they do not
know how to evaluate the risk specifi c to a transaction. Despite heavy news
coverage, people may have an illusion that hacking will not happen to them.
This illusion could explain why John Kelly, the former Secretary of Home-
land Security and White House chief of staff , used a compromised personal
phone for months.18
The third explanation is that consumers are fully aware of the risk, but
given the fact that their personal data has been shared with many fi rms and
has likely already been breached somewhere, they believe the extra risk of
sharing the data with one more organization is small. Survey evidence seems
to lend some support to this conjecture. According to the Pew Research
Center (2016), few are confi dent that the records of their activities main-
tained by various companies and organizations will remain private and
secure. A vast majority (91 percent) of adults agree that consumers have
lost control of how PII is collected and used by companies, though most
think personal control is important. Moreover, 86 percent of internet users
have taken steps to remove or mask their digital footprints, and many say
they would like to do more or are unaware of tools they could use.19
Consumer anxiety may explain why identity theft protection service has
become a $3 billion industry (according to IBISWorld).20 However, a market
review by the Government Accounting Offi
ce (GAO 2017) shows that iden-
tity theft services off er some benefi ts, but generally do not prevent identity
theft or address all of its variations. For instance, these services typically do
not address medical identity theft or identity theft refund fraud. In fact, a
number of identity theft service providers were caught making deceptive
marketing claims,21 casting doubt on whether such “insurance- like” services
are the best way to safeguard consumers from identity theft.
18. https:// www .wired .com/ story/ john- kelly- hacked- phone/ , accessed on October 15, 2017.
19. “The state of privacy in post- Snowden America” by the Pew Research Center, source: http:// www .pewresearch .org/ fact- tank/ 2016/ 09/ 21/ the- state- of-privacy- in-america/.
20. https:// www .ibisworld .com/ industry- trends/ specialized- market- research- reports
/ technology/ computer- services/ identity- theft- protection- services .html, accessed on October 26, 2017.
21. For example, in September 2012, Discover settled with the Consumer Financial Protection Bureau (CFPB) and the Federal Deposit Insurance Corporation (FDIC) with $200 mil-
lion refund to consumers and $14 million penalty. The CFPB and FDIC alleged that Discover engaged in misleading telemarketing on identity theft protection, credit score tracking, wallet
450 Ginger Zhe Jin
18.3.2 Supply Side Actions
Statistics from the supply side are mixed, too.
Thales (2017b) conducted a global survey of 1,100+ senior security execu-
tives, including 100+ respondents in key regional markets in the United
States, United Kingdom, Germany, Japan, Australia, Brazil, and Mexico,
and in key segments such as federal government, retail, fi nance, and health
care. It fi nds that 68 percent of survey respondents have ever experienced a
breach, while 26 percent experienced one last year. Both numbers rose from
2016 (61 percent and 22 percent).
For fi nancial services in particular, Thales (2017a) fi nds that fi rms are
aware of the cyber risk they face but tend to deploy new technology (e.g.,
cloud, big data, internet of things) before adopting security measures to
protect them. Only 27 percent of US fi nancial services organizations said
to feel “very” or “extremely” vulnerable to data threats (the global average
is 30 percent), despite the fact that 42 percent of US fi nancials had been
breached in the past (the global average is 56 percent). Consistently, both
US and global fi nancials rank data security at the bottom of their spending
plans, citing institutional inertia and complexity as the main reasons. These
numbers should be concerning because the fi nancial sector has the highest
cost of cyber crime according to the latest report from Accenture (2017).
To add a little comfort, Thales (2017a) also reports that security spending,
which includes but is not limited to data security, continues to trend up: 78
percent of US fi nancials reported higher spending than last year, trailing
only US health care (81 percent) and ahead of the overall global average
(73 percent).
Firms’ willingness to invest in data security is partially driven by the cost
they suff er directly from data breaches. A strand of literature has studied the
stock market’s response to data breach. While results diff er across studies,
the general fi nding is that the fi nancial market response is small and tempo-
rary, if negative at all (Campbell et al. 2003; Cavusoglu et al. 2004; Telang
and Wattal 2007; Ko and Dorantes 2006). A couple of studies have pro-
vided an absolute estimate of the cost. According to Ponemon (2017), who
surveyed 419 organizations in thirteen countries and regions, the average
consolidated total cost of a data breach is $3.62 million. Ponemon (2017)
further fi nds that data breaches are most expensive in the United States,
with the average per capita cost of data breach as high as $225. In contrast,
Romanosky (2016) examines a sample of 12,000 cyber events, including but
protection, and payment protection (http:// money.cnn .com/ 2012/ 09/ 24/ pf/ discover- penalty
- telemarketing/ index .html). In December 2015, LifeLock agreed to pay $100 million to settle FTC contempt charges for order violation. The 2010 court order requires the company to
secure consumers’ personal information and prohibits the company from deceptive advertising in identity theft protection services (https:// www .ftc .gov/ news- events/ press- releases/ 2015
/ 12/ lifelock- pay- 100-million- consumers- settle- ftc- charges- it- violated).
Artifi cial Intelligence and Consumer Privacy 451
not limited to data breaches. He fi nds that the cost of a typical cyber incident
(to the aff ected fi rm) is less than $200,000, roughly
0.4 percent of the fi rm’s
estimated annual revenues.
Thousands or millions, these estimates only refl ect the direct cost of the
cyber event to the fi rm, not all the consequential harm to consumers. For example, most breached fi rms off er one year of free credit monitoring service
for aff ected consumers, but data misuse can occur after a year. Either way,
consumers have to spend time, eff ort, and money to deal with identity theft,
reputation setback, fraud, blackmail, or even unemployment as a result of
a data breach. The lawsuit between the Federal Trade Commission (FTC)
and Wyndham Hotel and Resort gives a concrete example. Wyndham was
breached multiple times in 2008 and 2009, aff ecting more than 619,000
consumers. Before reaching a settlement, the FTC alleged that fraudulent
charges attributable to the Wyndham breaches exceeded $10.6 million.22
Although the fi nal settlement involves no money, this case suggests that
harm to consumers—via an increased risk of identity theft and the costs to
mediate the risk—can be much more substantial than the direct loss suff ered
by the breached fi rm. Arguably, it is this diff erence that motivates fi rms to
overcollect data or use lax data security, despite the real risk of data breach.
The good news is that market forces do push fi rms to respect consumer
demand for privacy and data security. For instance, Facebook profi les
expand over time and therefore the same default privacy setting tends to
reveal more personal information to larger audiences.23 In September 2014,
Facebook adjusted its default setting of privacy from public posting to
friend- only posting, which limits third party access to new users’ Facebook
posts. In the meantime, Facebook made it easier for existing users to update
their privacy settings, block out ads, and edit their ad profi les.24 We do not
know the exact reason behind the change, but it could be related to a few
things: for example, user willingness to share data on Facebook dropped
signifi cantly from 2005 to 2011 (Stutzman, Gross, and Acquisti 2012), aca-
demic research shows that it is very easy to identify strangers based on
photos publicly posted on Facebook (Acquisti, Gross, and Stutzman 2014),
and it costs Facebook $20 million to settle a class action lawsuit regarding
its “sponsored stories” (an advertising feature alleged to misappropriate