Among all forms of mistake, prophecy is the most gratuitous.
—GEORGE ELIOT, Middlemarch
What I’ve said that turned out to be right will be considered obvious, and what was wrong will be humorous.
— BILL GATES, The Road Ahead
IT IS COMMONPLACE to lament the sad state of political forecasting. Moreover, suspicions that the entire enterprise is intellectually bankrupt have only been fortified by the most recent forecasting fiasco: the unanimous declaration by quantitative modelers of presidential elections at the American Political Science Association in August 2000 that we could ignore the frantic rhetorical posturing of the next few months. Election campaigns are tales full of sound and fury but of no significance because of the offsetting effects of each side’s propaganda broadsides. The die had been cast: Gore would defeat Bush by decisive, even landslide, margins.1
We revisit this incident in chapter 5, so here it must suffice to caution against drawing sweeping conclusions from a single data point. The current chapter has three missions: (1) to explore why radical skeptics believe the social science quest for predictive laws to be ill-conceived; (2) to weave their arguments into a composite set of six hypotheses, the core tenets of skepticism, that tell us what to expect when a diverse array of experts tries to predict an even more diverse array of real-world events; (3) to present evidence that suggests that, although skepticism about the predictive powers of experts is warranted, the skeptics do sometimes overreach: “who gets what right” is not just a matter of blind luck.
RADICAL SKEPTICISM
Radical skeptics naturally gravitate toward a punctuated equilibrium view of politics.2 On the one hand, they must concede the obvious. Politics is sometimes drearily predictable. No expertise was required to know that war would not erupt in Scandinavia in the 1990s. In stable systems, we can often do well by relying on simple, predict-the-past algorithms. On the other hand, radical skeptics are keenly aware that all hell sometimes breaks loose. These bouts of severe unpredictability are, moreover, unpredictable, as unpredictable as the meteors that intermittently smash into our planet and radically alter the course of evolution, making—among other things—our branch of intelligent life possible.
Several of our more reluctant research participants suspected that unpredictability was more the rule than the exception in politics. Invoking Machiavelli, one cautioned that good (forecasting) judgment is more a matter of fortuna than of virtu.3 A second opined that Tolstoy had the “right take on great men”: those with reputations for farsightedness were lucky and how lucky becomes clear when we survey their mistakes as well as their triumphs. For example, Churchill gets credit for seeing the Nazi menace before almost everyone else, perhaps saving European Jews from total extermination, but he was not endowed with any preternatural gift. He may have merely had a lower threshold than others for seeing threats to British interests. After all, he did claim, in his campaign against self-government for India,4 to see ominous similarities between Gandhi and Hitler. A third puzzled over the paradoxes that arise in sizing up the judgment of that master practitioner of Realpolitik, Joseph Stalin. On the plus side of the amoral impact ledger, Stalin achieved total command of the Soviet Union and expanded Russian influence deeper into central Europe than any czar. On the minus side, he ignored warnings of an imminent Nazi invasion in 1941, attributing them to a British plot. We are thus left with a riddle: How could someone so pathologically paranoid on the home front have been so oblivious to the threat posed by a regime dedicated to annihilating “Judaeo-Bolshevism”?5 A fourth observed that even renowned speculators, such as George Soros, who brought the Bank of England to its knees in 1992, spotted the Thai baht’s weakness in 1997, and anticipated the Russian default of 1998, are eventually humbled. As Soros ruefully remarked on his “shorting” Internet stocks too soon, “We had our head handed to us.”6 Of course, Soros was “just off on timing.” The NASDAQ fell by 60 percent by 2001. Looking ahead, it remains to be seen whether the rampaging bulls of the late twentieth century have been set up to be mowed down in the early twenty-first century or the new-economy visionaries are right that things are different this time, and that Dow 36,000 is around the corner.
Skeptics also stress the fine line between success and failure. Churchill’s career was almost ruined in 1916 by his sponsorship of the disastrous Gallipoli campaign designed to knock the Ottoman Empire out of World War I. But Churchill insisted, and some historians agree, that the plan “almost worked” and would have if it had been more resolutely implemented.7 Conversely, Stalin arguably escaped his share of blame for his blunders because, in the end, he was victorious. Stalin nearly lost everything but was saved by Hitler’s even bigger blunders.
On close scrutiny, reputations for political genius rest on thin evidential foundations: genius is a matter of being in the right place at the right time. Hero worshippers reveal their own lack of historical imagination: their incapacity to see how easily things could have worked out far worse as a result of contingencies that no mortal could have foreseen. Political geniuses are just a close-call counterfactual away from being permanently pilloried as fools.
Varieties of Radical Skepticism
Figure 2.1 splits radical skeptics into two lines of intellectual descent: ontological skeptics who point to fundamental properties of the world that make it impossible to achieve forecasting accuracy beyond crude extrapolation algorithms and psychological skeptics who point to fundamental properties of the human mind that make it inevitable that experts will miss whatever predictability has not been precluded “in principle.”
ONTOLOGICAL SKEPTICS
This camp is populated by an odd assortment of path-dependency theorists, complexity theorists, game theorists, and probability theorists.
Path dependency. Polya’s urn is a simple game that makes a profound point: life can alternate—quite unpredictably—between periods of boring predictability and wild unpredictability.8 Players confront an urn with two balls, one red and one green. Players remove a ball, at random, and return it, plus an additional ball of the same color. And they repeat this procedure until they fill the urn. Polya urn processes have three defining characteristics:9 they are initially unpredictable (in the beginning, the final outcome could range anywhere from 99.9 percent red to .01 percent red), they become increasingly inflexible (later draws contribute only minutely to the final distribution), and they show how small initial advantages can quickly accumulate, making it hard to change direction.
Figure 2.1. The varied grounds that skeptics have for suspecting that observers will never be able to predict better than either chance or extrapolation algorithms. The more arguments one endorses, the more entrenched one’s skepticism toward the possibility of forecasting in complex social systems.
Path-dependency theorists argue that many historical processes should be modeled as quirky path-dependent games with the potential to yield increasing returns. They maintain that history has repeatedly demonstrated that a technology can achieve a decisive advantage over competitors even if it is not the best long-run alternative.10 These theorists have also not limited themselves to explaining the triumph of QWERTY typewriters, VHS recorders, and Microsoft Windows. They have locked bigger game into their explanatory sights.
The most ambitious application of increasing returns has been to the long-simmering controversy over “the rise of the West” (and the concomitant failure of the “Rest”). How did a comparative handful of Europeans, inhabiting a cultural backwater a thousand years ago, become the dominant force on the planet, reducing peoples on every other continent to tributary status?11 It was not obvious that Europe and its colonial offshoots would achieve global hegemony. China and Islam seemed like formidable contenders as late as A.D. 1300 or 1400. From an increasing-returns perspective, the key lies in the tiny advantages that Europe had in preconditions for growth: a fragile web of coevolving institutions that encouraged property rights and rule of law (giving entrepreneurs some protection from confiscation), a m
easured tolerance of free inquiry (facilitating a common pool of knowledge from which innovators could draw), market competition (rewarding ingenuity), and a competitive state system in which states that lagged economically soon faltered militarily. This synergistic combination underlies the exponential expansion of European influence that began around A.D. 1500 and, in a few centuries, propelled a laggard civilization ahead of its more sophisticated rivals.12
Not everyone, however, is sold on the wide applicability of increasing-returns, path-dependency views of history. Traditionalists subscribe to decreasing-returns approaches that portray both past and future as deducible from assumptions about how farsighted economic actors, working within material and political constraints, converge on unique equilibria. For example, Daniel Yergin notes how some oil industry observers in the early 1980s used a decreasing-returns framework to predict, thus far correctly, that OPEC’s greatest triumphs were behind it.13 They expected the sharp rises in oil prices in the late 1970s to stimulate conservation, exploration, and exploitation of other sources of energy, which would put downward pressure on oil prices. Each step from the equilibrium is harder than the last. Negative feedback stabilizes social systems because major changes in one direction are offset by counterreactions. Good judges appreciate that forecasts of prolonged radical shifts from the status quo are generally a bad bet.
Can forecasters tell us, ex ante, when to apply an increasing- or decreasing-returns framework? Skeptics doubt we can even make such determinations ex post: too much hinges on metaphysical guesswork. Who, this side of God, knows whether history has a diverging branching structure that leads to a variety of possible worlds, or a converging structure that, notwithstanding detours, channels us into destinations predetermined long ago?
Complexity Theorists. One could grant the ubiquity of path dependency but still embrace a moderate brand of skepticism that links good judgment to the ability to identify leverage points.14 A persistent mediator, such as Jimmy Carter at Camp David, might broker a peace that would otherwise have been lost, or a shrewd philanthropist, such as George Soros, might have a pretty good track record of picking projects that have impact disproportionate to the expenditure, such as photocopy machines for Soviet bloc countries or funds to pay unemployed Soviet scientists who might otherwise have sold their services to rogue states eager to obtain weapons of mass destruction.
Radical skeptics deny even this role for good judgment. Embracing complexity theory, they argue that history is a succession of chaotic shocks reverberating through incomprehensibly intricate networks. To back up this claim, they point to computer simulations of physical systems that show that, when investigators link well-established nonlinear relationships into positive feedback loops, tiny variations in inputs begin to have astonishingly large effects.15
McCloskey illustrates the point with a textbook problem of ecology: predicting how the population of a species next year will vary as a function of this year’s population.16 The model is xt + 1 = f(xt), a one-period-back nonlinear differential equation. The simplest equation is the hump: xt + 1 = βxt [1 – xt], where the tuning parameter, β, determines the hump’s shape by specifying how the population of deer at t + 1 depends on the population in the preceding period. More deer mean more reproductive opportunities, but more deer also exhaust the food supply and attract wolves. The higher β is, the steeper the hump and the more precipitous the shift from growth to decline. McCloskey shows how a tiny shift in beta from 3.94 to 3.935 can alter history. The plots of populations remain almost identical for several years but, for mysterious tipping-point reasons, the hypothetical populations decisively part ways twenty-five years into the simulation.
These tipping-point models are so compelling because they resonate so deeply with human experience. Who among us cannot imagine our lives unfolding differently but for tiny accidents of fate that shaped the jobs we hold, the people we marry, and so on? Counterfactual historians aggressively extend such “bifurcation point” arguments when they try to show it is “easy” to unravel not just the fates of individuals but also those of nations.17 One eminent practitioner of this genre, Robert Fogel, argues that, even as late as the 1850s, “the overarching role of contingent circumstances in the victory of the antislavery movement needs to be emphasized. There never was a moment between 1854 and 1860 in which the triumph of the anti-slavery coalition was assured.”18 And just as the Civil War was not foreordained, many historians insist that there was nothing inevitable about the war’s outcome. Accounts of military campaigns in the Civil War (and other wars) abound with tales of how—in the spirit of the nursery rhyme—horseshoe-nail-sized causes determined the outcomes of battles.19
We could endlessly multiply these examples of great oaks sprouting from little acorns. For radical skeptics, though, there is a deeper lesson: the impossibility of picking the influential acorns before the fact. Joel Mokyr compares searching for the seeds of the Industrial Revolution to “studying the history of Jewish dissenters between 50 A.D. and 50 B.C. We are looking for something that at its inception was insignificant, even bizarre, but destined to change the life of every man and woman in the West.”20
Academics often disdain such arguments. Butterfly effect arguments undercut their pet theories: wars break out not due to grand causes—primordial hatreds or power imbalances—but to petty ones—royal carriage drivers making wrong turns, giving astonished assassins, who had just botched their jobs earlier that day, second chances to do it right. There is not much scholarly panache in documenting cause-effect linkages of this sort, one triviality after another, no better than “journalism.”21 McCloskey, however, gets the last word: “The disdain for assigning large events small causes is not rational in a world that is partly non-linear.” If our fates are the products of extraordinary strings of coincidences, “it is ostrich-like foolishness to bury our heads in the sand and pretend that we live in a neatly deterministic and predictable world.”22
Game Theorists. The rivalry between Sherlock Holmes and the evil genius Professor Moriarty illustrates how indeterminacy can arise as a natural by-product of rational agents second-guessing each other. When the two first met, Moriarty was eager, too eager, to display his capacity for interactive thinking by announcing: “All I have to say has already crossed your mind.” Holmes replied: “Then possibly my answer has crossed yours.” As the plot unfolds, Holmes uses his superior “interactive knowledge” to outmaneuver Moriarty by unexpectedly getting off the train at Canterbury, thwarting Moriarty who had calculated that Paris was Holmes’s rational destination. Convoluted though it is, Moriarty failed to recognize that Holmes had already recognized that Moriarty would deduce what a rational Holmes would do under the circumstances, and the odds now favored Holmes getting off the train earlier than once planned.23
Indeterminacy problems of this sort are the bread and butter of behavioral game theory. In the “guess the number” game, for example, contestants pick a number between 0 and 100, with the goal of making their guess come as close as possible to two-thirds of the average guess of all the contestants.24 In a world of only rational players—who base their guesses on the maximum number of levels of deduction—the equilibrium is 0. However, in a contest run at Richard Thaler’s prompting by the Financial Times,25 the most popular guesses were 33 (the right guess if everyone else chooses a number at random, producing an average guess of 50) and 22 (the right guess if everyone thinks through the preceding argument and picks 33). Dwindling numbers of respondents carried the deductive logic to the third stage (picking two-thirds of 22) or higher, with a tiny hypereducated group recognizing the logically correct answer to be 0. The average guess was 18.91 and the winning guess, 13, which suggests that, for this newspaper’s readership, a third order of sophistication was roughly optimal.
But defenders of forecasting accuracy benchmarks of good judgment can argue that all is not lost, that we can model how people play such games by distinguishing two types of sophistication. Logically sophisticated but psy
chologically naïve players guess zero: they see the right answer but exaggerate how many others do. Logically and psychologically sophisticated players see the right answer and appreciate how many, or few, others also “get it.” Good judgment requires their mix of logical and psychological savvy.
Radical skeptics can counter, however, that many games have inherently indeterminate multiple or mixed strategy equilibria. They can also note that one does not need to buy into a hyperrational model of human nature to recognize that, when the stakes are high, players will try to second-guess each other to the point where political outcomes, like financial markets, resemble random walks.26 Indeed, radical skeptics delight in pointing to the warehouse of evidence that now attests to the unpredictability of the stock market. Burton Malkiel documents that most mutual funds did not outperform market averages over the past thirty years. He also finds little consistency in performance. Big winners in the 1970s often flopped in the 1980s and 1990s. Good judgment requires hanging in for the long haul (the random walk meanders around an upward trend) and resisting the siren calls of technical analysts who promise to divine the future from the entrails of past trends and of hot-tip market-timers who are oddly eager to share breathtaking opportunities with strangers.27
At this point, some readers will give skeptics a dose of their own medicine. Have the naysayers not heard of superstar investors such as Peter Lynch, Warren Buffett, and George Soros, who beat their competitors (and market averages) with eerie consistency? But die-hard skeptics are unfazed: they endorse the blasphemous thought that these objects of adulation just got lucky. The skeptics offer a striking analogy. Imagine tossing each of one hundred coins one hundred times. By chance, a small set will yield improbable streaks of heads or tails. Financial geniuses are statistical flukes—no more mysterious than a few coins landing heads five or ten consecutive times.
Expert Political Judgment Page 10