Overcomplicated

by Samuel Arbesman

  United States Code is far more complicated: Michael J. Bommarito II and Daniel M. Katz, “A Mathematical Approach to the Study of the United States Code,” Physica A: Statistical Mechanics and its Applications 389, no. 19 (2010): 4195–200, http://arxiv.org/abs/1003.4146.

  airplane the Wright brothers built: “Wright 1903 Flyer,” NASA Glenn Research Center, accessed June 17, 2015, http://wright.nasa.gov/airplane/air1903.html.

  A Boeing 747-400: “747 Fun Facts,” Boeing Commercial Airplanes, https://web.archive.org/web/20111205231111/http://www.boeing.com/commercial/747family/pf/pf_facts.html.

  the numbers of individual parts: Kelly, What Technology Wants, 279.

  Windows operating system became: Kelly, What Technology Wants, 279; David McCandless, “Codebases: Millions of Lines of Code,” infographic, v. 0.9, Information Is Beautiful, September 24, 2015, http://www.informationisbeautiful.net/visualizations/million-lines-of-code/.

  software application Photoshop: McCandless, “Codebases.”

  the American telephone system: M. D. Fagen, ed., A History of Engineering and Science in the Bell System: The Early Years (1875–1925), Technical Publication Department, Bell Laboratories, 1975.

  FAA began to examine its computers: The story of the FAA’s concern about Y2K is from Homer-Dixon, The Ingenuity Gap.

  representing the FAA technicians: Matthew L. Wald, “Warning Issued on Air Traffic Computers,” The New York Times, January 13, 1998, http://www.nytimes.com/1998/01/13/us/warning-issued-on-air-traffic-computers.html, accessed February 6, 2015. According to Homer-Dixon, one of the retired programmers was hired to help fix the problem.

  machines at the Internal Revenue Service: Anne Broache, “IRS Trudges On with Aging Computers,” CNET News, April 12, 2007. See also John Bodoh, “Tech Timebomb: The IRS Is Still Living in the 1960s,” Washington Examiner, December 17, 2014, http://www.washingtonexaminer.com/tech-timebomb-the-irs-is-still-living-in-the-1960s/article/2557483.

  final space shuttle mission: “The Shuttle: NASA’s IT Legacy,” Information Age, July 18, 2011, http://www.information-age.com/technology/applications-and-development/1641693/the-shuttle%3A-nasas-it-legacy.

  In The Mythical Man-Month: Frederick P. Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, anniversary ed. (Boston, MA: Addison-Wesley, 1995; orig. pub. 1975), 53. It seems that this quote is a Latin proverb, misattributed to Ovid.

  a process of accretion: This term is also used in Homer-Dixon, The Ingenuity Gap.

  “Typically, outdated legacy systems”: Stewart Brand, The Clock of the Long Now: Time and Responsibility (New York: Basic Books, 1999), 85.

  only gingerly poke it: From an essay by Stewart Brand: “Beyond the evanescence of data formats and digital storage media lies a deeper problem. Computer systems of large scale are at the core of driving corporations, public institutions, and indeed whole sectors of the economy. Over time, these gargantuan systems become dauntingly complex and unknowable, as new features are added, old bugs are worked around with layers of ‘patches,’ generations of programmers add new programming tools and styles, and portions of the system are repurposed to take on novel functions. With both respect and loathing, computer professionals call these monsters ‘legacy systems.’ Teasing a new function out of a legacy system is not done by command, but by conducting cautious alchemic experiments that, with luck, converge toward the desired outcome.” “Written on the Wind,” The Long Now Foundation, February 11, 1998, http://longnow.org/essays/written-wind/.

  pages of instructions for the 1040: Christopher Ingraham, “Charted: The Skyrocketing Complexity of the Federal Tax Code,” Washington Post, April 15, 2015, http://www.washingtonpost.com/blogs/wonkblog/wp/2015/04/15/charted-the-skyrocketing-complexity-of-the-federal-tax-code/.

  Supreme Court has ruled: In Cheek v. United States, 498 U.S 192. See also United States v. Murdock, 290 U.S. 389.

  pages in the Code of Federal Regulations: Susan E. Dudley and Jerry Brito, Regulation: A Primer, 2nd ed. (Arlington, VA, and Washington, DC: Mercatus Center, George Mason University, and The George Washington University Regulatory Studies Center, 2012), 5, http://mercatus.org/publication/regulation-primer.

  known as Parkinson’s Law: “Parkinson’s Law,” The Economist, November 19, 1955, accessed February 26, 2015, http://www.economist.com/node/14116121. This relationship does not seem to hold when a country is at war.

  those in the software world have enshrined this idea: See Meir M. Lehman, “Programs, Life Cycles, and Laws of Software Evolution,” Proceedings of the IEEE 68, no. 9 (1980): 1060–76. Also Lehman et al., “Metrics and Laws of Software Evolution—The Nineties View,” in METRICS ’97, Proceedings of the Fourth International Software Metrics Symposium (Washington, DC: IEEE Computer Society, 1997), 20–32. In addition, see The Systems Bible by John Gall, discussed later and in the Further Reading section.

  rewrite a piece of software: This rewriting is related to refactoring: recoding a piece of software so that internally it is a lot better and cleaner, even though the external functionality remains unchanged.

  trade-offs in time, effort, and money: Astronaut Alan Shepard famously said in a post-flight briefing, “It’s a very sobering feeling to be up in space and realize that one’s safety factor was determined by the lowest bidder on a government contract.”

  our cities have gas pipes: John Kelly, “Look Out Below: Danger Lurks Underground from Aging Gas Pipes,” USA Today, September 23, 2014, http://www.usatoday.com/story/news/nation/2014/09/23/gas-pipes-cast-iron-deaths-explosions-investigation/15783697/.

  run on 1930s technologies: Metropolitan Transportation Authority (MTA), “CBTC: Communications-Based Train Control,” July 20, 2015, https://www.youtube.com/watch?v=Mjx3S3UjmnA.

  systems become more complicated over time: Such systems even have to start off in a complex state, in order to anticipate edge cases and exceptions, as discussed later in the chapter.

  described as a “godsend”: Harry McCracken, “Fifty Years of BASIC, the Programming Language That Made Computers Personal,” TIME, April 29, 2014, http://time.com/69316/basic/. A more principled way of managing such branches and loops is to use more-explicit versions of them, such as “for loops.”

  but actually “harmful”: Edsger W. Dijkstra, “Go To Statement Considered Harmful,” Communications of the ACM 11, no. 3 (1968): 147–48. Note that Dijkstra’s original title for this paper was “A Case Against the GO TO Statement.”

  make sense of and impose order: Refactoring is one such method. Software developers Brian Foote and Joe Yoder have sought to explain why the “de-facto standard software architecture” is a “casually, even haphazardly structured system”—a “Big Ball of Mud”—and to map out ways to improve such systems from within. Brian Foote and Joseph Yoder, “Big Ball of Mud,” Fourth Conference on Pattern Languages of Programs, Monticello, IL, September 1997; in Pattern Languages of Program Design 4, ed. Brian Foote, Neil Harrison, and Hans Rohnert, chapter 29 (Boston: Addison-Wesley, 2000), available online: http://www.laputan.org/mud/.

  the software inside Toyota vehicles: Koopman, “Case Study of Toyota Unintended Acceleration,” slide 38. Koopman notes that spaghetti code can generate these high metrics of complexity. Practices such as the MISRA Software Guidelines, issued by The Motor Industry Software Reliability Association in the UK, have been developed to better ensure that these systems are safe.

  the story of the Bayonne Bridge: Philip K. Howard, The Rule of Nobody: Saving America from Dead Laws and Broken Government (New York: W. W. Norton, 2014).

  rules and regulations dictating the procedure: Howard, Rule of Nobody, 8.

  public projects taking around ten years: Howard, Rule of Nobody, 12. A project to replace another bridge took ten years to approve; the average duration of environmental review for highway projects is more than eight years.

  growth of rule systems as “
regulatory accumulation”: Michael Mandel and Diana G. Carew, Regulatory Improvement Commission: A Politically-Viable Approach to U.S. Regulatory Reform, Progressive Policy Institute policy memo (Washington, DC: Progressive Policy Institute, May 2013), available online: http://www.progressivepolicy.org/wp-content/uploads/2013/05/05.2013-Mandel-Carew_Regulatory-Improvement-Commission_A-Politically-Viable-Approach-to-US-Regulatory-Reform.pdf.

  concept of interoperability: John Palfrey and Urs Gasser, Interop: The Promise and Perils of Highly Interconnected Systems (New York: Basic Books, 2012).

  interdependence between different kinds of technologies: Sergey V. Buldyrev et al., “Catastrophic Cascade of Failures in Interdependent Networks,” Nature 464 (2010): 1025–28, http://polymer.bu.edu/hes/articles/bppsh10.pdf. See also Natalie Wolchover, “Treading Softly in a Connected World,” Quanta Magazine, March 18, 2013, https://www.quantamagazine.org/20130318-treading-softly-in-a-connected-world/.

  the cost of failure: Cost of failure is really a hypothetical distribution, not a single number, but it could be thought of in terms of an expected value or average.

  Northeast Blackout in 2003: J. R. Minkel, “The 2003 Northeast Blackout—Five Years Later,” Scientific American, August 13, 2008, http://www.scientificamerican.com/article/2003-blackout-five-years-later/.

  For a long time, this approach worked: These insights on the changes in cost of construction and cost of failure are thanks to Edward Jung (personal communication, March 28, 2014).

  Apple Maps mislabeled a supermarket: “A Maps App with Problems,” The New York Times, September 27, 2012, http://www.nytimes.com/slideshow/2012/09/27/technology/pogue-maps-ss-3.html.

  poliovirus has been reconstructed: Jeronimo Cello et al., “Chemical Synthesis of Poliovirus cDNA: Generation of Infectious Virus in the Absence of Natural Template,” Science 297 (2002): 1016–18; Eckard Wimmer, “The Test-Tube Synthesis of a Chemical Called Poliovirus: The Simple Synthesis of a Virus Has Far-Reaching Societal Implications,” EMBO Reports 7, no. 1S (2006): S3–S9, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490301/.

  a basic imperative of technology: For example, see Doug Hill, Not So Fast: Thinking Twice about Technology (Cellarius Press, 2013, out of print; Athens, GA: University of Georgia Press, forthcoming); Kevin Kelly, What Technology Wants.

  Dijkstra noted that programming a computer: Dijkstra, “On the Cruelty of Really Teaching Computing Science,” E. W. Dijkstra Archive: The manuscripts of Edsger W. Dijkstra, 1930–2002, document no. EWD1036, December 1988, http://www.cs.utexas.edu/users/EWD/ewd10xx/EWD1036.PDF [hand-printed original], http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1036.html [typed transcript].

  Systems we build to reflect the world: That the complexity of the world is reflected in the complexity of our systems is also discussed in Vikram Chandra, Geek Sublime: The Beauty of Code, the Code of Beauty (Minneapolis: Graywolf Press, 2014). One need not always end up with messy code because the world is messy, but it does often happen. Fortunately, there are ways to mitigate it. See Steve McConnell, Code Complete: A Practical Handbook of Software Construction, 2nd ed. (Redmond, WA: Microsoft Press, 2004), 583.

  building a self-driving vehicle: The complexity of building self-driving cars was discussed by Google[x]’s “Captain of Moonshots” in his closing keynote address at South by Southwest Interactive (SXSW) 2015: Astro Teller, “How to Make Moonshots,” Backchannel, March 17, 2015, https://medium.com/backchannel/how-to-make-moonshots-65845011a277.

  the exceptions that nonetheless have to be dealt with: One solution is to use humans to manually troubleshoot, or at least hard-code, the exceptions. For example, here’s how Google does this for Maps: “This is a Google-y approach to the problem of ultra-reliability. Many of Google’s famously computation driven projects—like the creation of Google Maps—employed literally thousands of people to supervise and correct the automatic systems. It is one of Google’s open secrets that they deploy human intelligence as a catalyst. Instead of programming in that last little bit of reliability, the final 1 or 0.1 or 0.01 percent, they can deploy a bit of cheap human brainpower. And over time, the humans work themselves out of jobs by teaching the machines how to act. ‘When the human says, “Here’s the right thing to do,” that becomes something we can bake into the system and that will happen slightly less often in the future,’ Teller said.” Alexis C. Madrigal, “Inside Google’s Secret Drone-Delivery Program,” The Atlantic, August 28, 2014, http://www.theatlantic.com/technology/archive/2014/08/inside-googles-secret-drone-delivery-program/379306/?single_page=true.

  Scholars think it might have been an error: “Friar Daw’s Reply,” from Six Ecclesiastical Satires, ed. James M. Dean, TEAMS Middle English Texts Series (Kalamazoo, MI: Medieval Institute Publications, 1991); available online at Robbins Library Digital Projects, University of Rochester, accessed April 30, 2015, http://d.lib.rochester.edu/teams/text/dean-six-ecclesiastical-satires-friar-daws-reply.

  more commonly, a long tail: Note that not all heavy-tailed distributions, or long tails, are necessarily power laws.

  Often about half of the words: András Kornai, Mathematical Linguistics (London: Springer-Verlag, 2008), 71. According to this source, the percentage of the words in a corpus that occur only once each—hapax legomena—is about 40–60 percent for many corpora.

  To avoid losing our exceptions and edge cases: Related ideas are explored, along with the notion of language as a complex system, in William A. Kretzschmar Jr., The Linguistics of Speech (Cambridge, UK: Cambridge University Press, 2009). Also related is Firthian linguistics, “based on the view that language patterns cannot be accounted for in terms of a single system of analytic principles and categories,” but that multiple different context-dependent systems may be called for. David Crystal, ed., Dictionary of Linguistics and Phonetics, 6th ed. (Malden, MA: Wiley-Blackwell, 2008), 181.

  Peter Norvig, Google’s director of research: Peter Norvig, “On Chomsky and the Two Cultures of Statistical Learning,” accessed April 30, 2015, http://norvig.com/chomsky.html.

  great, though apocryphal, story: There seem to be many versions of this apocryphal machine translation tale.

  What techniques are used by experts: Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford, UK: Oxford University Press, 2014), 15.

  say, 99.9 percent of the time: I made these numbers up for effect, but if any linguist wants to chat, please reach out!

  “based on millions of specific features”: Alon Halevy et al., “The Unreasonable Effectiveness of Data,” IEEE Intelligent Systems 24, no. 2 (2009): 8–12. In some ways, these statistical models are actually simpler than those that start from seemingly more elegant rules, because the latter end up being complicated by exceptions.

  sophisticated machine learning techniques: See Douglas Heaven, “Higher State of Mind,” New Scientist 219 (August 10, 2013), 32–35, available online (under the title “Not Like Us: Artificial Minds We Can’t Understand”): http://complex.elte.hu/~csabai/simulationLab/AI_08_August_2013_New_Scientist.pdf.

  Frederick P. Brooks Jr. has noted: Brooks, Mythical Man-Month, 183–84. Brooks recognizes many types of complexity, including that imposed by the environment the software must interact with.

  law turns out to look like a fractal: David G. Post and Michael B. Eisen, “How Long Is the Coastline of the Law? Thoughts on the Fractal Nature of Legal Systems,” Journal of Legal Studies 29, no. 2 J(2000): 545–84.

  legal scholar Jack Balkin discusses this: Jack M. Balkin, “The Crystalline Structure of Legal Thought,” Rutgers Law Review 39, no. 1 (1986): 1–108, http://www.yale.edu/lawweb/jbalkin/articles/crystal.pdf; Yale Law School Faculty Scholarship Series, Paper 294.

  The law professor David Post and the biologist Michael Eisen: Post and Eisen, “How Long Is the Coastline of the Law?”

  they find features indicative of fractals: Post and Eisen find power laws.

>   “the value of good contracts and good lawyering”: Mark D. Flood and Oliver Goodenough, “Contract as Automaton: The Computational Representation of Financial Agreements,” OFR (Office of Financial Research) Working Paper no. 15-04, March 26, 2015, https://financialresearch.gov/working-papers/files/OFRwp-2015-04_Contract-as-Automaton-The-Computational-Representation-of-Financial-Agreements.pdf.

  physics-trained sociologist Duncan Watts: Duncan Watts, “Too Complex to Exist,” The Boston Globe, June 14, 2009, http://www.boston.com/bostonglobe/ideas/articles/2009/06/14/too_complex_to_exist/?page=full.

  scholars have spoken of finding the optimal levels: Palfrey and Gasser, Interop. See, for example, the chapter on complexity.

  if each module in a system: Assuming that each distinct module can connect to another one in three distinct ways outbound and three distinct ways inbound, including self-loops, we have approximately 1032 potential networks. The number of stars in the observable universe is not known, but one estimate places it at fewer than 1030. Elizabeth Howell, “How Many Stars Are In The Universe?” Space.com, May 31, 2014, http://www.space.com/26078-how-many-stars-are-there.html.

  good computer science and engineering practices: While the main focus of this book is how to navigate through an age of incomprehensibility that is already upon us, there are ways to design and construct more-manageable engineered systems, such as using “systems thinking.” For example, see Nancy G. Leveson, Engineering a Safer World: Systems Thinking Applied to Safety (Cambridge, MA: The MIT Press, 2011).

  methods that can reduce the number of bugs: McConnell, Code Complete, 521.

  CHAPTER 3: LOSING THE BUBBLE

  In 1985, a patient entered a clinic: Story and analysis from Nancy G. Leveson and Clark S. Turner, “An Investigation of the Therac-25 Accidents,” Computer 26, no. 7 (1993), 18–41.

  “software does not degrade”: Quoted in Leveson and Turner, “An Investigation.”