b. If Bethmann-Hollweg had pressured Austro-Hungary more strongly not to declare war on Serbia, war would have been averted.
c. If Britain had clearly communicated to Germany its support of France in case of war, Germany would have exercised much more restraint on Austro-Hungary, thereby defusing the crisis.
d. If Germany had accepted Britain’s suggestion in late July of a great power conference to deal with the crisis and had pressured Austro-Hungary to do so too, war would been averted.
The close-call scenarios for Study 4 undid the outcomes of either World Wars I or II:
a. If Germany had proceeded with its invasion of France on August 2, 1914, but had respected the Belgian neutrality, Britain would not have entered the war and France would have quickly fallen.
b. If the German High Command had implemented the Schlieffen Plan more aggressively in August 1914, the miracle of the Marne would have been impossible and Paris would have fallen.
c. If Germany had avoided antagonizing the United States by meddling in Mexico and by initiating unrestricted submarine warfare, the United States would not have entered World War I and Germany would have prevailed against the French and British in its spring offensive of 1918.
d. If Hitler had not invaded the Soviet Union and concentrated German resources on defeating the British, Germany would have defeated Britain.
e. If Hitler had more consistently focused on taking Moscow in the summer of 1941, he could have knocked the Soviet Union out of the war.
f. If Hitler had not declared war on the United States on December 11, 1941, the British and the Soviets could never have defeated Nazi Germany.
The close-call counterfactuals for Study 5 explored the feasibility of turning the cold war into thermonuclear war:
a. If Stalin had lived several years longer (surviving his stroke but in an irrational state of mind that encouraged high-risk adventures), World War III could easily have broken out in the mid-1950s.
b. If bad weather had delayed the discovery by U-2 reconnaissance planes of Soviet missiles in Cuba until the missiles were operational, the Soviets would have refused American demands to dismantle and withdraw the weapons.
c. If the Soviets had refused to withdraw their missiles, the U.S. would have launched air strikes against the Soviet bases.
d. If the U.S. had launched such air strikes, the Soviet commanders in Cuba would have launched at least some missiles at the eastern seaboard of the United States.
e. If the Soviets had fired Cuban-based nuclear missiles at American cities, retaliatory nuclear strikes would have been launched at Soviet cities.
f. If Soviet hard-liners had taken charge of the Communist Party in the mid-1980s, the cold war—far from ending peacefully and quickly—coming to an early and peaceful end—would have intensified.
Study 6: Unmaking the West
PARTICIPANTS
The sixty-three participants were either randomly drawn from the membership roster of the World History Association or recruited from two conferences at the Mershon Center of the Ohio State University on the rise of the West. Respondents were contacted by either regular mail or by e-mail, and promised both anonymity and detailed feedback on the purposes of the study. The response rate was 31 percent. In addition to the measures described next, participants completed a nine-item abbreviated version of the need-for-closure scale in section I (items 1–9).
RESEARCH PROCEDURES AND MATERIALS
Covering-law Beliefs. The most relevant beliefs revolved around the theme of the survival-of-the-fittest civilizations:
a. History is, in the long run, an efficient process of winnowing out maladaptive forms of social organization.
b. Western societies and institutions, with their greater emphasis on the rule of law, property rights, free markets, and the practical applications of science, were better adapted to prevail in long-term competition with other civilizations.
Beliefs about Close-call Counterfactuals. The close-call counterfactuals explored the feasibility of unmaking the West via hypothetical interventions that either enfeebled European civilization or empowered rival civilizations:
a. If China had had, at key junctures, emperors more sympathetic to economic and technological development, it could have emerged as the world’s first superpower.
b. If the Mongols had continued their advance into central and western Europe and not been distracted by the death of Genghis Khan, later European development would have been impossible.
c. If Islamic armies had made a serious attempt to conquer France and Italy in the eighth century, later European development could have been radically sidetracked.
d. If the Black Death had been even more lethal, killing, say, 70 percent of the population, Europe could not have arisen as the dominant region in the second half of the millennium.
For each counterfactual, experts judged the following on nine-point scales:
a. How plausible was the antecedent condition of the argument? (Do we have to “rewrite” a little or a lot of history?)
b. Assuming the plausibility of the antecedent, how likely was the hypothesized consequence?
c. Assuming the plausibility of the hypothesized consequence, what would the long-term ramifications have been?
V. UNPACKING OF POSSIBLE FUTURES EXPERIMENTS (CHAPTER 7)
The goal here was to explore the impact of encouraging divergent scenario thinking about possible futures on both correspondence indicators of good judgment (e.g., forecasting accuracy) and coherence indicators (e.g., susceptibility to sub-additivity effects).
Participants and Context
These experiments were conducted roughly five weeks after the initial (baseline) subjective probability estimates were obtained in the regional forecasting exercises (section I) for Canada and for Japan (long enough for memories of their original answers to fade). Participants in the Canadian experiment included both experts on Canadian politics (n = 28) and dilettantes (n = 33). Participants in the Japanese experiment included both experts (n = 16) and dilettantes (n = 19).
As part of the rationale for recontacting them, we told participants there was growing interest in using “scenario methods for preparing for possible futures” and that we were soliciting experts’ reactions to a series of “stories about possible futures” of either Canada or Japan.
Research Procedures and Materials
UNPACKING POSSIBLE FUTURES OF CANADA
The Canadian experiment took the form of a 2 (experts or dilettantes) × 4 (timing of four measures) factorial that required participants to make four sets of judgments: the pre-scenario-exercise assessment of subjective probabilities, a second wave of assessments after doing the status quo scenario exercises, a third wave of assessments after doing the disintegration-of-Canada scenario exercises, and a fourth wave of reflective equilibrium assessments in which experts reconcile possible conflicts among the probabilities they assigned in earlier exercises and ensure those probabilities sum to 1.0.
The stories fell into two broad categories: (a) possible futures that featured either a continuation of the status quo (federal and provincial governments agree to continue disagreeing over constitutional prerogatives) or a strengthening of Canadian unity (in which some agreements are reached); (b) possible futures in which Canada unravels, a secessionist referendum in Quebec succeeds, and controversy centers on terms of the divorce.
The continuation-of-Canadian-confederation scenarios solicited likelihood judgments of four possible futures. In the first, the combination of an economic downturn and skillful handling of federal-provincial relations by the prime minister leads (risk-averse) voters in Quebec to hand defeat to the Parti Québecois (PQ) in the next election. In a second scenario, the same antecedents are at work and the Parti Québecois wins the next election but loses the secessionist referendum because voters are reluctant to take big risks in hard times. In a third scenario, the combination of an economic upturn and skillful handling of federal-provincial relations by the prim
e minister leads to the defeat of the Parti Québecois in the next election. In a fourth scenario, the same antecedents are at work and the Parti Québecois wins the next election but loses the secessionist referendum because voters are reluctant to gamble with their new prosperity. Experts were also invited to consider all other possible futures consistent with either a continuation or strengthening of Canadian unity. Experts then (a) assigned subjective likelihoods to each of the four possible futures as well as to the fifth residual category; and (b) rated the “ease of imagining” each possible future.
The disintegration scenario presented stories about futures in which Quebec secedes from Canada. These stories again split into four versions. In one, an economic downturn plus a confrontational prime minister who provokes separatist sentiment (with remarks about francophone “linguistic fascism”) lead the PQ to victory in both the next election and in the secessionist referendum (the poor economy increases voters’ willingness to take risks), and the resulting divorce is surprisingly amicable. In the second, the same antecedents are in place, and the result remains the same (PQ victory in next election and secessionist referendum), but the divorce is acrimonious. In the third and fourth scenarios, all is the same except now there is an economic upturn, Quebec still secedes (the strong economy gives voters confidence that they can “go it alone”), and the divorce is either easy or hard. Again, experts (a) judged the subjective likelihood of the four sets of possible futures as well as that of a residual category containing all other secessionist possibilities; and (b) rated the ease of imagining each story line.
UNPACKING POSSIBLE FUTURES OF JAPAN
The Japanese experiment took the form of a 2 (experts or dilettantes) × 5 (timing of repeated measures) design. The five-level factor included a baseline assessment of the subjective probabilities of three sets of possible futures (status quo and change for either better or worse), assessment of the same possible futures after unpacking each of the three sets, and finally a reflective equilibrium exercise in which participants confronted and tried to resolve logical contradictions created by the unpacking exercises.
In the Japanese scenario study, the perpetuation-of-status-quo set of possibilities included these two subclasses: (a) the ruling Liberal Democratic Party (LDP) maintains its electoral lock on power and resists the economic reforms and restructuring necessary to jumpstart growth but makes sufficient accommodations to reality to prevent serious deterioration and a slide into a deep, prolonged recession; (b) the LDP fractures and briefly loses power to a reformist coalition that begins to implement some politically painful economic reform, loses popularity, falls from power, and the LDP regains power and returns to policies that maintain the status quo. The improvement-on-status-quo set included these two subsets: (a) a reformist faction in the LDP gains power and implements politically painful economic reform; the government becomes unpopular during the transitional period but succeeds in laying down a new legal-financial infrastructure that will be conducive to growth in the future; (b) the LDP disintegrates into factions and a reformist coalition of parties takes power and implements the policies necessary to create a legal-financial infrastructure that will support more robust future growth. The deterioration-relative-to-status-quo set included these two subsets: (a) special interests inside the LDP patronage network succeed not only in blocking reform but also in thwarting essential accommodations to financial reality (e.g., writing off bad bank loans) and, as a result, Japan suffers a protracted slump; (b) the LDP loses power, but the weak coalition and minority successor governments lack the political support and economic wisdom to implement necessary reform and, as a result, Japan enters a protracted slump. As in the Canadian study, we allowed for residual categories into which participants could dump the likelihood of all other possible pathways to the specified superordinate classes of outcomes.
VI. UNPACKING-OF-HISTORICAL-COUNTERFACTUALS EXPERIMENTS (CHAPTER 7)
The goal here was to assess the impact of divergent scenario-based thinking about possible pasts on the hindsight bias (Studies 1 and 2) and on judgments of how quickly historical outcomes became (inevitability curves) or alternative outcomes became impossible (impossibility curves) (Studies 3 and 4).
The Cognitive Style (Hedgehog-Fox) Measure in this Set of Studies
Respondents gave answers on a nine-point agree-disagree scale to the following nine items: (a) “I think that having clear rules and order at work is essential for success”; (b) “Even after I have made up my mind about something, I am always eager to consider a different opinion”; (c) “I dislike questions that can be answered in many different ways”; (d) “I usually make important decisions quickly and confidently”; (e) “When considering most conflict situations, I can usually see how both sides could be right”; (f) “I prefer interacting with people whose opinions are very different from my own”; (g) “When trying to solve a problem I often see so many possible options that it is confusing”; (h) “Scholars are usually at greater risk of exaggerating the complexity of political processes than they are of underestimating the complexity of those processes”; (i) “Isaiah Berlin [1997] classified intellectuals as hedgehogs or foxes. A hedgehog knows one big thing and tries to integrate the diversity of the world into comprehensive and parsimonious vision whereas a fox knows many small things and tries to improvise explanations on a case-by-case basis. I would place myself toward the hedgehog or fox style of thinking about politics.”
Participants in Studies 1 and 2
Participants were drawn from the regional forecasting exercises for China and North Korea reported in chapter 7. The studies were conducted in 1997–1998 in the context of the recontact interviews for the belief-updating exercises described in section II. Sample sizes were twenty-one and fourteen, respectively.
Research Procedures
We noted in Chapter 4 that in six of the eleven belief-updating exercises, we asked participants to recall their earlier predictions before we reminded them of what those predictions actually were and before we asked them to respond to the Taking Stock Questionnaire (results described in chapter 4). In two of these six exercises, we informed participants—after they had tried to recall their predictions but before we reminded them of the truth—of the following: “Looking back on what has happened in China/North Korea over the last five years, we would value your expert opinion on how close we came to experiencing alternative outcomes—alternative outcomes that are either significantly better politically or economically than the current situation or significantly worse. What specific scenarios come to mind?” Given the timing and context of the exercise, it was important to create a social atmosphere in which (a) experts felt they would not lose face if they revised their recollections so soon after just expressing them; (b) experts did not, however, feel pressured to change their recollections in a direction that would confirm the experimental hypothesis that counterfactual scenario exercises help to check hindsight bias. Accordingly, we also offered the following additional information: “There is a lot of controversy surrounding the usefulness of scenario exercises of this sort. Some say such exercises give us a keener appreciation of how the world looked before we knew how things would work out. Others say such exercises give us a keener appreciation for why things had to work out as they did. Still others say such exercises rarely cause anyone to change his or her mind. We are doing this research, in part, to find out which of these ideas is right.”
Participants then sketched counterfactual alternatives to reality for roughly twenty minutes (responses that we could code for number of distinct scenarios, valence of scenarios [better or worse worlds], and amount of detail within and across scenarios). After this exercise, we then asked participants: “In light of the exercise you just did, do you want to modify any of your earlier estimates of the subjective probabilities that you assigned five years ago?”
Participants in Studies 3 and 4
Respondents were initially sampled from faculty at two midwestern universities (pilot groups
), but for the Cuban missile crisis study, they were subsequently randomly selected from the membership lists of Divisions 18 and 19 (International Conflict; International Security and Arms Control) of the APSA and the Society for Historians of American Foreign Relations, and for the Rise of the West study, they were randomly sampled from the membership roster of the World History Association.
Respondents in the missile crisis study completed (a) a nine-item version of the need-for-closure scale (items 1–9 in section I); (b) a measure of faith in the robustness of nuclear deterrence (same as used in Study 5 in section IV). Respondents in the Unmaking the West study completed the same nine-item need-for-closure scale and the survival-of-the-fittest civilizations scale used in Study 6 in section IV.
Research Procedures for the Cuban Missile Crisis Experiment (Study 3)
The 64 subjects were randomly assigned to: (a) a no-unpacking control group (n = 30), which responded to the “perceptions-of-inevitability” scale for judging the actual peaceful outcome of the crisis and to the “perceptions-of-impossibility” scale for judging alternative more violent endings of the crisis; (b) an unpacking-of-alternative-violent-outcomes conditions (n = 34) in which, before judging anything, participants unpacked the set of alternatives into those in which violence is localized or spreads outside the Caribbean, then unpacked those subsets into those in which violence claims less or more than 100 casualties, and then finally unpacked the subsets with 100 or more casualties into those in which only conventional weaponry is used and those in which one or both superpowers employed nuclear weapons. Respondents then judged the “imaginability” of each of the six sets of scenarios (nine-point scale: easy–difficult) and generated both inevitability curves for peace and impossibility curves for war. Order of elicitation of inevitability and impossibility judgments was always counterbalanced.
Expert Political Judgment Page 42