by Hannah Fry
31 Sharon Bertsch McGrayne, The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy (New Haven: Yale University Press, 2011).
32 M. Bayes and M. Price, An Essay towards Solving a Problem in the Doctrine of Chances. By the Late Rev. Mr. Bayes, F.R.S. Communicated by Mr. Price, in a Letter to John Canton, A.M.F.R.S. (1763), digital copy uploaded to archive.org 2 Aug. 2011, https://archive.org/details/philtrans09948070.
33 Michael Taylor, ‘Self-driving Mercedes-Benzes will prioritize occupant safety over pedestrians’, Car and Driver, 7 Oct. 2016, https://blog.caranddriver.com/self-driving-mercedes-will-prioritize-occupant-safety-over-pedestrians/.
34 Jason Kottke, Mercedes’ Solution to the Trolley Problem, Kottke.org, 24 Oct. 2016, https://kottke.org/16/10/mercedes-solution-to-the-trolley-problem.
35 Jean-François Bonnefon, Azim Shariff and Iyad Rahwan (2016), ‘The social dilemma of autonomous vehicles’, Science, vol. 35, 24 June 2016, DOI 10.1126/science.aaf2654; https://arxiv.org/pdf/1510.03346.pdf.
36 All quotes from Paul Newman are from private conversation.
37 Naaman Zhou, ‘Volvo admits its self-driving cars are confused by kangaroos’, Guardian, 1 July 2017, https://www.theguardian.com/technology/2017/jul/01/volvo-admits-its-self-driving-cars-are-confused-by-kangaroos.
38 All quotes from Jack Stilgoe are from private conversation.
39 Jeff Sabatini, ‘The one simple reason nobody is talking realistically about driverless cars’, Car and Driver, Oct. 2017, https://www.caranddriver.com/features/the-one-reason-nobody-is-talking-realistically-about-driverless-cars-feature.
40 William Langewiesche, ‘The human factor’, Vanity Fair, 17 Sept. 2014, https://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash.
41 Bureau d’Enquêtes et d’Analyses pour la Sécuritié de l’Aviation Civile, Final Report on the Accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France Flight AF447 Rio de Janeiro – Paris, Eng. edn (Paris, updated July 2012), https://www.bea.aero/docspa/2009/f-cp090601.en/pdf/f-cp090601.en.pdf.
42 Ibid.
43 Langewiesche, ‘The human factor’.
44 Ibid.
45 Jeff Wise, ‘What really happened aboard Air France 447’, Popular Mechanics, 6 Dec. 2011, http://www.popularmechanics.com/flight/a3115/what-really-happened-aboard-air-france-447-6611877/.
46 Langewiesche, ‘The human factor’.
47 Wise, ‘What really happened aboard Air France 447’.
48 Lisanne Bainbridge, ‘Ironies of automation’, Automatica, vol. 19, no. 6, Nov. 1983, pp. 775–9, https://www.sciencedirect.com/science/article/pii/0005109883900468.
49 Ibid.
50 Alex Davies, ‘Everyone wants a level 5 self-driving car – here’s what that means’, Wired, 26 July 2016.
51 Justin Hughes, ‘Car autonomy levels explained’, The Drive, 3 Nov. 2017, http://www.thedrive.com/sheetmetal/15724/what-are-these-levels-of-autonomy-anyway.
52 Bainbridge, ‘Ironies of automation’.
53 Jack Stilgoe, ‘Machine learning, social learning and the governance of self-driving cars’, Social Studies of Science, vol. 48, no. 1, 2017, pp. 25–56.
54 Eric Tingwall, ‘Where are autonomous cars right now? Four systems tested’, Car and Driver, Oct. 2017, https://www.caranddriver.com/features/where-are-autonomous-cars-right-now-four-systems-tested-feature.
55 Tracey Lindeman, ‘Using an orange to fool Tesla’s autopilot is probably a really bad idea’, Motherboard, 16 Jan. 2018, https://motherboard.vice.com/en_us/article/a3na9p/tesla-autosteer-orange-hack.
56 Daisuke Wakabayashi, ‘Uber’s self-driving cars were struggling before Arizona Crash’, New York Times, 23 March 2018, https://www.nytimes.com/2018/03/23/technology/uber-self-driving-cars-arizona.html.
57 Sam Levin, ‘Video released of Uber self-driving crash that killed woman in Arizona’, Guardian, 22 March 2018, https://www.theguardian.com/technology/2018/mar/22/video-released-of-uber-self-driving-crash-that-killed-woman-in-arizona.
58 Audi, The Audi vision of autonomous driving, Audi Newsroom, 11 Sept. 2017, https://media.audiusa.com/en-us/releases/184.
59 P. Morgan, C. Alford and G. Parkhurst, Handover Issues in Autonomous Driving: A Literature Review. Project Report (Bristol: University of the West of England, June 2016), http://eprints.uwe.ac.uk/29167/1/Venturer_WP5.2Lit%20ReviewHandover.pdf.
60 Langewiesche, ‘The human factor’.
61 Evan Ackerman, ‘Toyota’s Gill Pratt on self-driving cars and the reality of full autonomy’, IEEE Spectrum, 23 Jan. 2017, https://spectrum.ieee.org/cars-that-think/transportation/self-driving/toyota-gill-pratt-on-the-reality-of-full-autonomy.
62 Julia Pyper, ‘Self-driving cars could cut greenhouse gas pollution’, Scientific American, 15 Sept. 2014, https://www.scientificamerican.com/article/self-driving-cars-could-cut-greenhouse-gas-pollution/.
63 Raphael E. Stern et al., ‘Dissipation of stop-and-go waves via control of autonomous vehicles: field experiments’, arXiv: 1705.01693v1, 4 May 2017, https://arxiv.org/abs/1705.01693.
64 SomeJoe7777, ‘Tesla Model S forward collision warning saves the day’, YouTube, 19 Oct. 2016, https://www.youtube.com/watch?v=SnRp56XjV_M.
65 Jordan Golson and Dieter Bohn, ‘All new Tesla cars now have hardware for “full self-driving capabilities”: but some safety features will be disabled initially’, The Verge, 19 Oct. 2016, https://www.theverge.com/2016/10/19/13340938/tesla-autopilot-update-model-3-elon-musk-update.
66 Fred Lambert, ‘Tesla introduces first phase of “Enhanced Autopilot”: “measured and cautious for next several hundred million miles” – release notes’, Electrek, 1 Jan 2017, https://electrek.co/2017/01/01/tesla-enhanced-autopilot-release-notes/.
67 DPC Cars, ‘Toyota Guardian and Chauffeur autonomous vehicle platform’, YouTube, 27 Sept. 2017, https://www.youtube.com/watch?v=IMdceKGJ9Oc.
68 Brian Milligan, ‘The most significant development since the safety belt’, BBC News, 15 April 2018, http://www.bbc.co.uk/news/business-43752226.
Crime
1 Bob Taylor, Crimebuster: Inside the Minds of Britain’s Most Evil Criminals (London: Piatkus, 2002), ch. 9, ‘A day out from jail’.
2 Ibid.
3 Nick Davies, ‘Dangerous, in prison – but free to rape’, Guardian, 5 Oct. 1999, https://www.theguardian.com/uk/1999/oct/05/nickdavies1.
4 João Medeiros, ‘How geographic profiling helps find serial criminals’, Wired, Nov. 2014, http://www.wired.co.uk/article/mapping-murder.
5 Nicole H. Rafter, ed., The Origins of Criminology: A Reader (Abingdon: Routledge, 2009), p. 271.
6 Luke Dormehl, The Formula: How Algorithms Solve All Our Problems … and Create More (London: W. H. Allen, 2014), p. 117.
7 Dormehl, The Formula, p. 116.
8 D. Kim Rossmo, ‘Geographic profiling’, in Gerben Bruinsma and David Weisburd, eds, Encyclopedia of Criminology and Criminal Justice (New York: Springer, 2014), https://link.springer.com/referenceworkentry/10.1007%2F978-1-4614-5690-2_678.
9 Ibid.
10 João Medeiros, ‘How geographic profiling helps find serial criminals’.
11 Ibid.
12 ‘“Sadistic” serial rapist sentenced to eight life terms’, Independent (Ireland), 6 Oct. 1999, http://www.independent.ie/world-news/sadistic-serial-rapist-sentenced-to-eight-life-terms-26134260.html.
13 Ibid.
14 Steven C. Le Comber, D. Kim Rossmo, Ali N. Hassan, Douglas O. Fuller and John C. Beier, ‘Geographic profiling as a novel spatial tool for targeting infectious disease control’, International Journal of Health Geographics, vol. 10, no.1, 2011, p. 35, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3123167/.
15 Michelle V. Hauge, Mark D. Stevenson, D. Kim Rossmo and Steven C. Le Comber, ‘Tagging Banksy: using geographic profiling to investigate a modern art mystery’, Journal of Spatial Science, vol. 61, no. 1, 2016, pp. 185–90, http://www.tandfonline.
com/doi/abs/10.1080/14498596.2016.1138246.
16 Raymond Dussault, ‘Jack Maple: betting on intelligence’, Government Technology, 31 March 1999, http://www.govtech.com/featured/Jack-Maple-Betting-on-Intelligence.html.
17 Ibid.
18 Ibid.
19 Nicole Gelinas, ‘How Bratton’s NYPD saved the subway system’, New York Post, 6 Aug. 2016, http://nypost.com/2016/08/06/how-brattons-nypd-saved-the-subway-system/.
20 Dussault, ‘Jack Maple: betting on intelligence’.
21 Andrew Guthrie Ferguson, ‘Predictive policing and reasonable suspicion’, Emory Law Journal, vol. 62, no. 2, 2012, p. 259, http://law.emory.edu/elj/content/volume-62/issue-2/articles/predicting-policing-and-reasonable-suspicion.html.
22 Lawrence W. Sherman, Patrick R. Gartin and Michael E. Buerger, ‘Hot spots of predatory crime: routine activities and the criminology of place’, Criminology, vol. 27, no. 1, 1989, pp. 27–56, http://onlinelibrary.wiley.com/doi/10.1111/j.1745-9125.1989.tb00862.x/abstract.
23 Toby Davies and Shane D. Johnson, ‘Examining the relationship between road structure and burglary risk via quantitative network analysis’, Journal of Quantitative Criminology, vol. 31, no. 3, 2015, pp. 481–507, http://discovery.ucl.ac.uk/1456293/5/Johnson_art%253A10.1007%252Fs10940-014-9235-4.pdf.
24 Michael J. Frith, Shane D. Johnson and Hannah M. Fry, ‘Role of the street network in burglars’ spatial decision-making’, Criminology, vol. 55, no. 2, 2017, pp. 344–76, http://onlinelibrary.wiley.com/doi/10.1111/1745-9125.12133/full.
25 Spencer Chainey, Predictive Mapping (Predictive Policing), JDI Brief (London: Jill Dando Institute of Security and Crime Science, University College London, 2012), http://discovery.ucl.ac.uk/1344080/3/JDIBriefs_PredictiveMappingSChaineyApril2012.pdf
26 Ibid.
27 A slight disclaimer. The PredPol algorithm itself isn’t publicly available. The experiment we’re referring to here was conducted by the same mathematicians who founded PredPol, using a technique that matches up to how the proprietary software is described. All the clues suggest they’re the same thing, but strictly speaking we can’t be absolutely sure.
28 G. O. Mohler, M. B. Short, Sean Malinowski, Mark Johnson, G. E. Tita, Andrea L. Bertozzi and P. J. Brantingham, ‘Randomized controlled field trials of predictive policing’, Journal of the American Statistical Association, vol. 110, no. 512, 2015, pp. 1399–1411, http://www.tandfonline.com/doi/abs/10.1080/01621459.2015.1077710.
29 Kent Police Corporate Services Analysis Department, PredPol Operational Review, 2014, http://www.statewatch.org/docbin/uk-2014-kent-police-predpol-op-review.pdf.
30 Mohler et al., ‘Randomized controlled field trials of predictive policing’.
31 Kent Police Corporate Services Analysis Department, PredPol Operational Review: Initial Findings, 2013, https://www.whatdotheyknow.com/request/181341/response/454199/attach/3/13%2010%20888%20Appendix.pdf.
32 Kent Police Corporate Services Analysis Department, PredPol Operational Review.
33 This wasn’t actually PredPol, but a much simpler algorithm that also used the ideas of the ‘flag’ and ‘boost’ effects. See Matthew Fielding and Vincent Jones, ‘Disrupting the optimal forager: predictive risk mapping and domestic burglary reduction in Trafford, Greater Manchester’, International Journal of Police Science and Management, vol. 14, no. 1, 2012, pp. 30–41.
34 Joe Newbold, ‘“Predictive policing”, “preventative policing” or “intelligence led policing”. What is the future?’ Consultancy project submitted in assessment for Warwick MBA programme, Warwick Business School, 2015.
35 Data from 2016: COMPSTAT, Citywide Profile 12/04/16–12/31/16, http://assets.lapdonline.org/assets/pdf/123116cityprof.pdf.
36 Ronald V. Clarke and Mike Hough, Crime and Police Effectiveness, Home Office Research Study no. 79 (London: HMSO, 1984), https://archive.org/stream/op1276605-1001/op1276605-1001_djvu.txt, as told in Tom Gash, Criminal: The Truth about Why People Do Bad Things (London: Allen Lane, 2016).
37 Kent Police Corporate Services Analysis Department, PredPol Operational Review.
38 PredPol, ‘Recent examples of crime reduction’, 2017, http://www.predpol.com/results/.
39 Aaron Shapiro, ‘Reform predictive policing’, Nature, vol. 541, no. 7638, 25 Jan. 2017, http://www.nature.com/news/reform-predictive-policing-1.21338.
40 Chicago Data Portal, Strategic Subject List, https://data.cityofchicago.org/Public-Safety/Strategic-Subject-List/4aki-r3np.
41 Jessica Saunders, Priscilla Hunt and John Hollywood, ‘Predictions put into practice: a quasi-experimental evaluation of Chicago’s predictive policing pilot’, Journal of Experimental Criminology, vol. 12, no. 3, 2016, pp. 347–71.
42 Copblock, ‘Innocent man arrested for robbery and assault, spends two months in Denver jail’, 28 April 2015, https://www.copblock.org/122644/man-arrested-for-robbery-assault-he-didnt-commit-spends-two-months-in-denver-jail/.
43 Ibid.
44 Ava Kofman, ‘How a facial recognition mismatch can ruin your life’, The Intercept, 13 Oct. 2016.
45 Ibid.
46 Copblock, ‘Denver police, “Don’t f*ck with the biggest gang in Denver” before beating man wrongfully arrested – TWICE!!’, 30 Jan. 2016, https://www.copblock.org/152823/denver-police-fck-up-again/.
47 Actually, Talley’s story is even more harrowing than the summary I’ve given here. After spending two months in jail for his initial arrest, Talley was released without charge. It was a year later – by which time he was living in a shelter – that he was arrested for a second time. This time the charges were not dropped, and the FBI testified against him. The case against him finally fell apart when the bank teller, who realized that Talley lacked the moles she’d seen on the robber’s hands as they passed over the counter, testified in court: ‘It’s not the guy who robbed me.’ He’s now suing for $10 million. For a full account, see Kofman, ‘How a facial recognition mismatch can ruin your life’.
48 Justin Huggler, ‘Facial recognition software to catch terrorists being tested at Berlin station’, Telegraph, 2 Aug. 2017, http://www.telegraph.co.uk/news/2017/08/02/facial-recognition-software-catch-terrorists-tested-berlin-station/.
49 David Kravets, ‘Driver’s license facial recognition tech leads to 4,000 New York arrests’, Ars Technica, 22 Aug. 2017, https://arstechnica.com/tech-policy/2017/08/biometrics-leads-to-thousands-of-a-ny-arrests-for-fraud-identity-theft/.
50 Ruth Mosalski, ‘The first arrest using facial recognition software has been made’, Wales Online, 2 June 2017, http://www.walesonline.co.uk/news/local-news/first-arrest-using-facial-recognition-13126934.
51 Sebastian Anthony, ‘UK police arrest man via automatic face recognition tech’, Ars Technica, 6 June 2017, https://arstechnica.com/tech-policy/2017/06/police-automatic-face-recognition.
52 David White, Richard I. Kemp, Rob Jenkins, Michael Matheson and A. Mike Burton, ‘Passport officers’ errors in face matching’, PLOSOne, 18 Aug. 2014, http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0103510#s6.
53 Teghan Lucas and Maciej Henneberg, ‘Are human faces unique? A metric approach to finding single individuals without duplicates in large samples’, Forensic Science International, vol. 257, Dec. 2015, pp. 514e1–514.e6, http://www.sciencedirect.com/science/article/pii/S0379073815003758.
54 Zaria Gorvett, ‘You are surprisingly likely to have a living doppelganger’, BBC Future, 13 July 2016, http://www.bbc.com/future/story/20160712-you-are-surprisingly-likely-to-have-a-living-doppelganger.
55 ‘Eyewitness misidentification’, The Innocence Project, https://www.innocenceproject.org/causes/eyewitness-misidentification.
56 Douglas Starr, ‘Forensics gone wrong: when DNA snares the innocent’, Science, 7 March 2016, http://www.sciencemag.org/news/2016/03/forensics-gone-wrong-when-dna-snares-innocent.
57 This doesn’t mean that false identifications are out of the question with DNA – they do happen; it just means you have a weapon on your side to make them as rare as possible.
>
58 Richard W. Vorder Bruegge, Individualization of People from Images (Quantico, Va., FBI Operational Technology Division, Forensic Audio, Video and Image Analysis Unit), 12 Dec. 2016, https://www.nist.gov/sites/default/files/documents/2016/12/12/vorderbruegge-face.pdf.
59 Lance Ulanoff, ‘The iPhone X can’t tell the difference between twins’, Mashable UK, 31 Oct. 2017, http://mashable.com/2017/10/31/putting-iphone-x-face-id-to-twin-test/#A87kA26aAqqQ.
60 Kif Leswing, ‘Apple says the iPhone X’s facial recognition system isn’t for kids’, Business Insider UK, 27 Sept. 2017, http://uk.businessinsider.com/apple-says-the-iphone-xs-face-id-is-less-accurate-on-kids-under-13-2017-9.
61 Andy Greenberg, ‘Watch a 10-year-old’s face unlock his mom’s iPhone X’, Wired, 14 Nov. 2017, https://www.wired.com/story/10-year-old-face-id-unlocks-mothers-iphone-x/.
62 ‘Bkav’s new mask beats Face ID in “twin way”: severity level raised, do not use Face ID in business transactions’, Bkav Corporation, 27 Nov. 2017, http://www.bkav.com/dt/top-news/-/view_content/content/103968/bkav%EF%BF%BDs-new-mask-beats-face-id-in-twin-way-severity-level-raised-do-not-use-face-id-in-business-transactions.
63 Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer and Michael Reiter, ‘Accessorize to a crime: real and stealthy attacks on state-of-the-art face recognition’, paper presented at ACM SIGSAC Conference, 2016, https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf.
64 Ira Kemelmacher-Shlizerman, Steven M. Seitz, Daniel Miller and Evan Brossard, The MegaFace Benchmark: 1 Million Faces for Recognition at Scale, Computer Vision Foundation, 2015, https://arxiv.org/abs/1512.00596
65 ‘Half of all American adults are in a police face recognition database, new report finds’, press release, Georgetown Law, 18 Oct. 2016, https://www.law.georgetown.edu/news/press-releases/half-of-all-american-adults-are-in-a-police-face-recognition-database-new-report-finds.cfm.
66 Josh Chin and Liza Lin, ‘China’s all-seeing surveillance state is reading its citizens’ faces’, Wall Street Journal, 6 June 2017, https://www.wsj.com/articles/the-all-seeing-surveillance-state-feared-in-the-west-is-a-reality-in-china-1498493020.