Data and Goliath

by Bruce Schneier

  It doesn’t mean collect: The DoD even cautions against thinking about and using words accurately. “Procedure 2 introduces the reader of DoD 5240.1-R to his or her first entry into the ‘maze’ of the regulation. To begin the journey, it is necessary to stop first and adjust your vocabulary. The terms and words used in DoD 5240.1-R have very specific meanings, and it is often the case that one can be led astray by relying on the generic or commonly understood definition of a particular word.” US Defense Intelligence Agency, Defense HUMINT Service (Aug 2004), Intelligence Law Handbook, Defense Intelligence Management Document CC-0000-181-95, https://www.aclu.org/files/assets/eo12333/DIA/Intelligence%20Law%20Handbook%20Defense%20HUMINT%20Service.pdf.

  All those books are stored: Andrea Mitchell (9 Jun 2013), “Transcript of Andrea Mitchell’s interview with Director of National Intelligence James Clapper,” NBC News, http://www.nbcumv.com/mediavillage/networks/nbcnews/pressreleases?pr=contents/press-releases/2013/06/09/nbcnewsexclusiv1370799482417.xml.

  Clapper asserts he didn’t lie: Ron Wyden (12 Mar 2013), “Wyden in intelligence hearing on GPS surveillance & Nat’l Security Agency collection,” YouTube, https://www.youtube.com/watch?v=QwiUVUJmGjs.

  no human reads those Gmail messages: Google (2014), “Ads in Gmail,” https://support.google.com/mail/answer/6603?hl=en.

  You might be told: In 2010, the TSA assured us that its full-body scanners were not saving data. Documents released to the Electronic Privacy Information Center showed that the scanners were shipped with hard drives and USB ports. Ginger McCall (3 Aug 2010), “Documents reveal that body scanners routinely store and record images,” Electronic Privacy Information Center, http://epic.org/press/EPIC_Body_Scanner_Press_Release_08_03_10.pdf. Declan McCullagh (4 Aug 2010), “Feds admit storing checkpoint body scan images,” CNET, http://www.cnet.com/news/feds-admit-storing-checkpoint-body-scan-images. US Transportation Security Administration (6 Aug 2010), “TSA response to ‘Feds admit storing checkpoint body scan images,’” TSA Blog, http://blog.tsa.gov/2010/08/tsa-response-to-feds-admit-storing.html.

  The primary difference: This is why we’re not worried about Furbies, but would be if they contained recording devices. Although for a while, the NSA was worried. British Broadcasting Corporation (13 Jan 1999), “Furby toy or Furby spy?” BBC News, http://news.bbc.co.uk/2/hi/americas/254094.stm.

  If you do object: Bruce Schneier (21 Oct 2013), “Why the NSA’s defense of mass data collection makes no sense,” Atlantic, http://www.theatlantic.com/politics/archive/2013/10/why-the-nsas-defense-of-mass-data-collection-makes-no-sense/280715.

  The means to perform identification: Bruce Schneier (2000), Secrets and Lies: Digital Security in a Networked World, Wiley, chap. 9, http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471453803.html.

  We can’t even be sure: Charles Glaser (1 Jun 2011), “Deterrence of cyber attacks and U.S. national security,” Report GW-CSPRI-2011-5, George Washington University Cyber Security Policy and Research Institute, http://www.cspri.seas.gwu.edu/uploads/2/1/3/2/21324690/2011-5_cyber_deterrence_and_security_glaser.pdf. Joseph S. Nye Jr. (May 2010), “Cyber power,” Harvard Kennedy School, Belfer Center for Science and International Affairs, http://belfercenter.ksg.harvard.edu/files/cyber-power.pdf.

  The 2007 cyberattack against Estonia: Charles Clover (11 Mar 2009), “Kremlin-backed group behind Estonia cyber blitz,” Financial Times, http://www.ft.com/cms/s/0/57536d5a-0ddc-11de-8ea3-0000779fd2ac.html. Christian Love (12 Mar 2009), “Kremlin loyalist says launched Estonia cyber-attack,” Reuters, http://www.reuters.com/article/2009/03/12/us-russia-estonia-cyberspace-idUSTRE52B4D820090312.

  It took analysts months: Nicole Perlroth (31 Jan 2013), “Hackers in China attacked the Times for last 4 months,” New York Times, http://www.nytimes.com/2013/01/31/technology/chinese-hackers-infiltrate-new-york-times-computers.html.

  who was behind Stuxnet: William J. Broad, John Markoff, and David E. Sanger (15 Jan 2011), “Israeli test on worm called crucial in Iran nuclear delay,” New York Times, http://www.nytimes.com/2011/01/16/world/middleeast/16stuxnet.html. David E. Sanger (1 Jun 2012), “Obama order sped up wave of cyberattacks against Iran,” New York Times, http://www.nytimes.com/2012/06/01/world/middleeast/obama-ordered-wave-of-cyberattacks-against-iran.html.

  proposals to eliminate anonymity: Limiting anonymity doesn’t eliminate trolls. People’s behavior online is complicated, and more a function of the loosening of social restrictions than of anonymity. John Suler (Jun 2004), “The online disinhibition effect,” Cyber Psychology and Behavior 7, http://online.liebertpub.com/doi/abs/10.1089/1094931041291295.

  annoys countries like China: Philipp Winter and Stefan Lindskog (6 Aug 2012), “How the Great Firewall of China is blocking Tor,” Second USENIX Workshop on Free and Open Communications on the Internet, Bellevue, Washington, https://www.usenix.org/system/files/conference/foci12/foci12-final2.pdf.

  Leon Panetta said publicly: Leon Panetta (11 Oct 2012), “Remarks by Secretary Panetta on cybersecurity to the Business Executives for National Security, New York City,” US Department of Defense, http://www.defense.gov/transcripts/transcript.aspx?transcriptid=5136.

  11: Security

  we tend to focus on rare: Bruce Schneier (17 May 2007), “Virginia Tech lesson: Rare risks breed irrational responses,” Wired, http://archive.wired.com/politics/security/commentary/securitymatters/2007/05/securitymatters_0517.

  we fear terrorists more: Washington’s Blog (15 Aug 2014), “You’re nine times more likely to be killed by a police officer than a terrorist,” Washington’s Blog, http://www.washingtonsblog.com/2014/08/youre-nine-times-likely-killed-police-officer-terrorist.html.

  connect-the-dots metaphor: Spencer Ackerman (13 Dec 2013), “NSA review to leave spying programs largely unchanged, reports say,” Guardian, http://www.theguardian.com/world/2013/dec/13/nsa-review-to-leave-spying-programs-largely-unchanged-reports-say.

  That doesn’t stop us: When we look back at an event and see all the evidence, we often believe we should have connected the dots. There’s a name for that: hindsight bias. The useful bits of data are obvious after the fact, but were only a few items in a sea of millions of irrelevant data bits beforehand. And those data bits could have been assembled to point in a million different directions.

  the “narrative fallacy”: Nassim Nicholas Taleb (2007), “The narrative fallacy,” in The Black Swan: The Impact of the Highly Improbable, Random House, chap. 6, http://www.fooledbyrandomness.com.

  The TSA’s no-fly list: Associated Press (2 Feb 2012), “U.S. no-fly list doubles in one year,” USA Today, http://usatoday30.usatoday.com/news/washington/story/2012-02-02/no-fly-list/52926968/1.

  the watch list: Eric Schmitt and Michael S. Schmidt (24 Apr 2013), “2 U.S. agencies added Boston bomb suspect to watch list,” New York Times, https://www.nytimes.com/2013/04/25/us/tamerlan-tsarnaev-bomb-suspect-was-on-watch-lists.html.

  Detecting credit card fraud: E. W. T. Ngai et al. (Feb 2011), “The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature,” Decision Support Systems 50, https://www.sciencedirect.com/science/article/pii/S0167923610001302. Siddhartha Bhattacharyya et al. (Feb 2011), “Data mining for credit card fraud: A comparative study,” Decision Support Systems 50, https://www.sciencedirect.com/science/article/pii/S0167923610001326.

  a billion active credit cards: Erika Harrell and Lynn Langton (12 Dec 2013), “Victims of identity theft 2012,” US Bureau of Justice Statistics, http://www.bjs.gov/index.cfm?ty=pbdetail&iid=4821.

  the IRS uses data mining: US Government Accountability Office (2013), “Offshore tax evasion: IRS has collected billions of dollars, but may be missing continued evasion,” Report GAO-13-318, http://www.gao.gov/assets/660/653369.pdf. IBM Corporation (2011), “New York State Tax: How predictive modeling improves tax revenues and citizen equity,” https://www.ibm.com/smarterplanet/us/en/leadership/nystax/assets/pdf/0623-NYS-Tax_Paper.pdf.

  the police use it: Walter L. Pe
rry et al. (2013), “Predictive policing: The role of crime forecasting in law enforcement operations,” RAND Corporation, https://www.ncjrs.gov/pdffiles1/nij/grants/243830.pdf.

  Terrorist plots are different: John Mueller and Mark G. Stewart (2011), Terror, Security, and Money: Balancing the Risks, Benefits, and Costs of Homeland Security, Oxford University Press, chap. 2, http://books.google.com/books?id=jyYGL2jZBC4C.

  even highly accurate . . . systems: Jeff Jonas and Jim Harper (11 Dec 2006), “Effective counterterrorism and the limited role of predictive data mining,” Cato Institute, http://www.cato.org/publications/policy-analysis/effective-counterterrorism-limited-role-predictive-data-mining. Fred H. Cate (Summer 2008), “Government data mining: The need for a legal framework,” Harvard Civil Rights-Civil Liberties Law Review 43, http://www.law.harvard.edu/students/orgs/crcl/vol43_2/435-490_Cate.pdf.

  false positives completely overwhelm: G. Stuart Mendenhall and Mark Schmidhofer (Winter 2012-13), “Screening tests for terrorism,” Regulation, http://object.cato.org/sites/cato.org/files/serials/files/regulation/2013/1/v35n4-4.pdf. Corey Chivers (6 Jun 2013), “How likely is the NSA PRISM program to catch a terrorist?” Bayesian Biologist, http://bayesianbiologist.com/2013/06/06/how-likely-is-the-nsa-prism-program-to-catch-a-terrorist. Marcy Wheeler (15 Jun 2013), “The inefficacy of Big Brother: Associations and the terror factory,” Empty Wheel, http://www.emptywheel.net/2013/06/15/the-inefficacy-of-big-brother-associations-and-the-terror-factory.

  millions of people will be falsely accused: In statistics, this is called the base rate fallacy, and it applies in other domains as well. For example, even highly accurate medical tests are problematic as screening tools if the incidence of the disease is sufficiently rare in the general population. I am deliberately not walking you through the math. Those who are interested can read the details. Jeff Jonas and Jim Harper (11 Dec 2006), “Effective counterterrorism and the limited role of predictive data mining,” Cato Institute, http://object.cato.org/sites/cato.org/files/pubs/pdf/pa584.pdf.

  “you need the haystack”: J. D. Tuccille (19 Jul 2013), “Why spy on everybody? Because ‘you need the haystack to find the needle,’ says NSA chief,” Reason, http://reason.com/blog/2013/07/19/why-spy-on-everybody-because-you-need-th.

  adding much more noise: Mike Masnick (15 Oct 2013), “Latest revelations show how collecting all the haystacks to find the needle makes the NSA’s job harder,” Tech Dirt, https://www.techdirt.com/articles/20131014/17303424880/latest-revelations-show-how-collecting-all-haystacks-to-find-data-makes-nsas-job-harder.shtml.

  so much irrelevant data: Chris Young (12 Mar 2012), “Military intelligence redefined: Big Data in the battlefield,” Forbes, http://www.forbes.com/sites/techonomy/2012/03/12/military-intelligence-redefined-big-data-in-the-battlefield.

  NSA’s eavesdropping program: Matt Briggs (7 Jun 2013), “Data mining: PRISM, NSA and false positives: Update,” William M. Briggs, http://wmbriggs.com/blog/?p=8239.

  thousands of tips: Lowell Bergman et al. (17 Jan 2006), “Spy agency data after Sept. 11 led F.B.I. to dead ends,” New York Times, http://www.nytimes.com/2006/01/17/politics/17spy.html.

  Suspicious Activity Reports: US Government Accountability Office (26 Mar 2013), “Information sharing: Additional actions could help ensure that efforts to share terrorism-related suspicious activity reports are effective,” Report GAO-13-233, http://www.gao.gov/assets/660/652995.pdf.

  led to just one success: Yochai Benkler (8 Oct 2013), “Fact: The NSA gets negligible intel from Americans’ metadata. So end collection,” Guardian, http://www.theguardian.com/commentisfree/2013/oct/08/nsa-bulk-metadata-surveillance-intelligence. Peter Bergen (Jan 2014), “Do NSA’s bulk surveillance programs stop terrorists?” New America Foundation, http://newamerica.net/publications/policy/do_nsas_bulk_surveillance_programs_stop_terrorists.

  that was probably trumped up: Marcy Wheeler (12 Dec 2013), “Did DOJ prosecute Basaaly Moalin just to have a Section 215 ‘success’?” Empty Wheel, http://www.emptywheel.net/2013/12/12/did-doj-prosecute-basaaly-moalin-just-to-have-a-section-215-success.

  Each rare individual: Airplane security provides many examples. In 2001, Richard Reid put a bomb in his shoe, and the primary effect is that we’ve all had to take our shoes off at airports since then.

  Several analyses: Francis Gouillart (10 Jun 2013), “Big data NSA spying is not even an effective strategy,” Fortune, http://management.fortune.cnn.com/2013/06/10/big-data-nsa-spying-is-not-even-an-effective-strategy. Ed Pilkington and Nicholas Watt (12 Jun 2013), “NSA surveillance played little role in foiling terror plots, experts say,” Guardian, http://www.theguardian.com/world/2013/jun/12/nsa-surveillance-data-terror-attack. Washington’s Blog (13 Jun 2013), “The dirty little secret about mass surveillance: It doesn’t keep us safe,” Washington’s Blog, http://www.washingtonsblog.com/2013/06/the-dirty-little-secret-about-nsa-spying-it-doesnt-work.html.

  Data mining is simply the wrong tool: Jeffrey W. Seifert (3 Apr 2008), “Data mining and homeland security: An overview,” Congressional Research Service, http://www.fas.org/sgp/crs/homesec/RL31798.pdf.

  enabled the NSA to prevent 9/11: Peter Bergen (30 Dec 2013), “Would NSA surveillance have stopped 9/11 plot?” CNN, http://www.cnn.com/2013/12/30/opinion/bergen-nsa-surveillance-september-11.

  wasn’t able to prevent: Simon Shuster (19 Apr 2013), “The brothers Tsarnaev: Clues to the motives of the alleged Boston bombers,” Time, http://world.time.com/2013/04/19/the-brothers-tsarnaevs-motives.

  The NSA collected data: Marcy Wheeler (12 Apr 2014), “The day after government catalogs data NSA collected on Tsarnaevs, DOJ refuses to give Dzhokhar notice,” Empty Wheel, http://www.emptywheel.net/2014/04/12/the-day-after-government-catalogs-data-nsa-collected-on-tsarnaevs-doj-refuses-to-give-dzhokhar-notice.

  failures were the result: National Commission on Terrorist Attacks (2004), The 9/11 Commission Report: Final Report of the National Commission on Terrorist Activities upon the United States, http://www.gpo.gov/fdsys/pkg/GPO-911REPORT/pdf/GPO-911REPORT.pdf.

  Mass surveillance didn’t catch: Dan Eggen, Karen DeYoung, and Spencer S. Hsu (27 Dec 2009), “Plane suspect was listed in terror database after father alerted U.S. officials,” Washington Post, http://www.washingtonpost.com/wp-dyn/content/article/2009/12/25/AR2009122501355.html.

  the liquid bombers . . . were captured: Dominic Casciani (7 Sep 2009), “Liquid bomb plot: What happened,” BBC News, http://news.bbc.co.uk/2/hi/uk_news/8242479.stm.

  comes from targeted surveillance: The NSA has touted 54 terrorist successes, but this number doesn’t pass scrutiny. Most weren’t actually terrorist plots, and they were mostly outside the US. Justin Elliott and Theodoric Meyer (23 Oct 2013), “Claim on ‘attacks thwarted’ by NSA spreads despite lack of evidence,” Pro Publica, http://www.propublica.org/article/claim-on-attacks-thwarted-by-nsa-spreads-despite-lack-of-evidence.

  FBI identifies potential terrorist plots: Kevin Strom and John Hollywood (2010), “Building on clues: Examining successes and failures in detecting U.S. terrorist plots,” Institute for Homeland Security Solutions, http://sites.duke.edu/ihss/files/2011/12/Building_on_Clues_Strom.pdf.

  the money we’re wasting: Bruce Schneier (8 Sep 2005), “Terrorists don’t do movie plots,” Wired, http://archive.wired.com/politics/security/commentary/securitymatters/2005/09/68789.

  the attacker has the advantage: Bruce Schneier (2012), Liars and Outliers: Enabling the Trust That Society Needs to Thrive, Wiley, chap. 16, http://www.wiley.com/WileyCDA/WileyTitle/productCd-1118143302.html.

  It’s easier to break things: Ross Anderson (2 Oct 2001), “Why information security is hard: An economic perspective,” University of Cambridge Computer Laboratory, http://www.acsac.org/2001/papers/110.pdf. Matthew Miller, Jon Brickey, and Gregory Conti (29 Nov 2012), “Why your intuition about cyber warfare is probably wrong,” Small Wars Journal, http://smallwarsjournal.com/jrnl/art/why-your-intuition-about-cyber-warfare-is-probably-wrong.

  Complexity is the worst enemy: Bruce
Schneier (19 Nov 1999), “A plea for simplicity: You can’t secure what you don’t understand,” Information Security, https://www.schneier.com/essay-018.html.

  Software security is generally poor: Edward Tufte (2003), “Why producing good software is difficult,” Edward Tufte Forum, http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0000D8. James Kwak (8 Aug 2012), “Software runs the world: How scared should we be that so much of it is so bad?” Atlantic, http://www.theatlantic.com/business/archive/2012/08/software-runs-the-world-how-scared-should-we-be-that-so-much-of-it-is-so-bad/260846.

  retailer Target Corporation: Michael Riley et al. (13 Mar 2014), “Missed alarms and 40 million stolen credit card numbers: How Target blew it,” Bloomberg Businessweek, http://www.businessweek.com/articles/2014-03-13/target-missed-alarms-in-epic-hack-of-credit-card-data.

  a catastrophe for the company: Elizabeth A. Harris et al. (17 Jan 2014), “A sneaky path into Target customers’ wallets,” New York Times, http://www.nytimes.com/2014/01/18/business/a-sneaky-path-into-target-customers-wallets.html.

  its CEO, Gregg Steinhafel, resigned: Elizabeth A. Harris (6 May 2014), “Faltering Target parts ways with chief,” New York Times, http://www.nytimes.com/2014/05/06/business/target-chief-executive-resigns.html.

  Compare this with the: Nicole Perlroth (31 Jan 2013), “Hackers in China attacked the Times for last 4 months,” New York Times, http://www.nytimes.com/2013/01/31/technology/chinese-hackers-infiltrate-new-york-times-computers.html.

  Multiprogram Research Facility: Its current goal is exaflop computation speeds, or one quintillion operations per second. James Bamford (15 Mar 2012), “The NSA is building the country’s biggest spy center (watch what you say),” Wired, http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/all.