Thinking in Bets

by Annie Duke

  When they got to White Castle, Ira the Whale decided to order the burgers twenty at a time. David knew he was a lock to win as soon as Ira the Whale ordered the first twenty, because Ira the Whale also ordered a milkshake and fries.

  After finishing the 100 burgers and after he and David collected their bets, Ira the Whale ordered another twenty burgers to go, “for Mrs. Whale.”

  Accountability is a willingness or obligation to answer for our actions or beliefs to others. A bet is a form of accountability. If we’re in love with our own opinions, it can cost us in a bet. Ira the Whale held the other gamblers accountable for their beliefs about whether he could eat 100 White Castle burgers. Accountability is why John Hennigan (briefly) moved to Des Moines. After spending time in that kind of environment, you become hypervigilant about your level of confidence in your beliefs. No one is forced to make or take such bets, but the prospect is a reminder that you can always be held accountable for the accuracy of what you believe and say. It is truly putting your money where your mouth is.

  Being in an environment where the challenge of a bet is always looming works to reduce motivated reasoning. Such an environment changes the frame through which we view disconfirming information, reinforcing the frame change that our truthseeking group rewards. Evidence that might contradict a belief we hold is no longer viewed through as hurtful a frame. Rather, it is viewed as helpful because it can improve our chances of making a better bet. And winning a bet triggers a reinforcing positive update.

  Accountability, like reinforcement of accuracy, also improves our decision-making and information processing when we are away from the group because we know in advance that we will have to answer to the group for our decisions. Early in my poker career, my poker group recommended that a way to avoid the effects of self-serving bias when I was losing was to have a preset “loss limit”—if I lost $600 at the stakes I was playing, I would leave the game. The smart, experienced players advising me knew that in the moment of losing, I might not be my most rational self in assessing whether I was losing because I was getting unlucky or losing because I was playing poorly. A predetermined loss limit acts as a check against irrationally chasing losses, but self-enforcement is a problem. If you have more money in your pocket, you might still take it out. If you’re out of money, casinos have ATMs and machines that let you get cash advances on your credit cards. Poker players are also pretty liberal about lending money to losing players.

  I was much less likely to break a loss limit because I knew I was accountable to my pod. If I reached my loss limit and my inner voice said, “This game is so good that I should put up more money and keep playing,” it also reminded me I’d have to answer for the decision to a group of players I respected. Accountability made me run that conversation in my head, in which I started explaining how I was just getting unlucky and they would expose why I was likely biased in my assessment, helping me resist the urge to buy more chips. And, after leaving a losing game and going home, I could offset some of the sting of losing by running the conversation where my pod would approve of my decision to quit the game when I told them about it.

  Imagining how the discussion will go helps us to spot more errors on our own and catch them more quickly.

  The group ideally exposes us to a diversity of viewpoints

  John Stuart Mill is one of the heroes of thinking in bets. More than one hundred and fifty years after writing On Liberty, his thinking on social and political philosophy remains startlingly current. One of the frequent themes in On Liberty is the importance of diversity of opinion. Diversity and dissent are not only checks on fallibility, but the only means of testing the ultimate truth of an opinion: “The only way in which a human being can make some approach to knowing the whole of a subject, is by hearing what can be said about it by persons of every variety of opinion, and studying all modes in which it can be looked at by every character of mind. No wise man ever acquired his wisdom in any mode but this; nor is it in the nature of human intellect to become wise in any other manner.”

  There is a simple beauty in Mill’s insight. On our own, we have just one viewpoint. That’s our limitation as humans. But if we take a bunch of people with that limitation and put them together in a group, we get exposed to diverse opinions, can test alternative hypotheses, and move toward accuracy. It is almost impossible for us, on our own, to get the diversity of viewpoints provided by the combined manpower of a well-formed decision pod. To get a more objective view of the world, we need an environment that exposes us to alternate hypotheses and different perspectives. That doesn’t apply only to the world around us: to view ourselves in a more realistic way, we need other people to fill in our blind spots.

  A group with diverse viewpoints can help us by sharing the work suggested in the previous two chapters to combat motivated reasoning about beliefs and biased outcome fielding. When we think in bets, we run through a series of questions to examine the accuracy of our beliefs. For example:

  Why might my belief not be true?

  What other evidence might be out there bearing on my belief?

  Are there similar areas I can look toward to gauge whether similar beliefs to mine are true?

  What sources of information could I have missed or minimized on the way to reaching my belief?

  What are the reasons someone else could have a different belief, what’s their support, and why might they be right instead of me?

  What other perspectives are there as to why things turned out the way they did?

  Just by asking ourselves these questions, we are taking a big step toward calibration. But there is only so much we can do to answer these questions on our own. We only get exposed to the information we have been exposed to, only live the experiences we have experienced, only think of the hypotheses that we can conceive of. It’s hard to know what reasons someone else could have for believing something different. We aren’t them. We haven’t had their experiences. We don’t know what different information they have. But they do.

  Much of our biased information processing stems from the amount of rope that uncertainty affords us. Well-deployed diversity of viewpoints in a group can reduce uncertainty due to incomplete information by filling in the gaps in what we know, making life start to fit more neatly on a chessboard.

  Others aren’t wrapped up in preserving our narrative, anchored by our biases. It is a lot easier to have someone else offer their perspective than for you to imagine you’re another person and think about what their perspective might be. A diverse group can do some of the heavy lifting of de-biasing for us. A poker table is a naturally diverse setting because we generally don’t select who we play with for their opinions. Even better, when there is disagreement stemming from the diverse opinions represented at a poker table, the discussion may naturally progress toward betting on it. These are ideal circumstances for promoting accuracy.

  Numerous groups have recognized the need to engineer the kind of diversity and encouragement of dissent that naturally occurs at a poker table. The State Department, since the Vietnam War, has had a formal Dissent Channel, where employees can have their dissenting views heard and addressed without fear of penalty. The American Foreign Service Association, the professional organization of foreign-service employees, has four separate awards it gives annually to members “to recognize and encourage constructive dissent and risk-taking in the Foreign Service.” The Dissent Channel has been credited with a policy change that helped end the genocidal war in Bosnia. In June 2016, fifty-one State Department employees signed a memo calling for President Obama to strengthen American military efforts in Syria. In late January 2017, approximately one thousand employees signed a dissent cable in response to President Trump’s executive order suspending immigration from seven Muslim-majority countries. The Dissent Channel represents something hopeful in our nation’s decision-making process. In an environment of increased polarization, foreign-service employees can m
ake their voices heard about policies with which they disagree, and do it regardless of whether the administration is Democrat or Republican. Allowing dissent has a value that transcends party politics.

  After September 11, the CIA created “red teams” that, according to Georgetown law professor Neal Katyal in a New York Times op-ed, “are dedicated to arguing against the intelligence community’s conventional wisdom and spotting flaws in logic and analysis.” Senior Obama administration officials, following the raid that killed Osama bin Laden, mentioned red-team analysis among the methods used to measure the degree of confidence that bin Laden, in the absence of visual or auditory confirmation, was in the compound subject to the raid.

  Dissent channels and red teams are a beautiful implementation of Mill’s bedrock principle that we can’t know the truth of a matter without hearing the other side. This commitment to diversity of opinion is something that we would be wise to apply to our own decision groups. For example, if a corporate strategy group is figuring out how to integrate operations following a merger, someone who initially opposed the merger would be good to have as part of the group. Perhaps they have reasons why the two sales departments won’t mesh—whatever their reasons, they could help the majority move forward with a wiser approach by taking those reasons into account.

  Diversity is the foundation of productive group decision-making, but we can’t underestimate how hard it is to maintain. We all tend to gravitate toward people who are near clones of us. After all, it feels good to hear our ideas echoed back to us. If there is any doubt about how easy it can be to fall into this confirmatory drift, we can even see this tendency in groups we consider some of the most dedicated to truthseeking: judges and scientists.

  Federal judges: drift happens

  Cass Sunstein, now a Harvard law professor, conducted a massive study with colleagues when he was on the faculty at the University of Chicago Law School, on ideological diversity in federal judicial panels. Sunstein recognized at the outset that the U.S. Courts of Appeals are “an extraordinary and longstanding natural experiment” in diversity. Appellate court panels are composed of three judges randomly drawn from that circuit’s pool. Each circuit’s pool includes life-tenured judges chosen (when an opening occurs or Congress recognizes the need for additional judges) by the sitting president. In any particular appeal, you could get a panel of three Democrat appointees, three Republican appointees, or a two-to-one mix in either direction.

  The study, encompassing over 6,000 federal appeals and nearly 20,000 individual votes, found, not surprisingly, that judicial voting generally followed political lines. Pure, unaided open-mindedness, even by life-tenured judges sworn to uphold the law, is hard.

  When there was political diversity on the panels, the researchers found several areas where that diversity improved the panel’s work. Even though, in most cases, two politically similar judges could dictate the panel’s outcome, there were significant differences between heterogeneous and homogeneous panels. A single panelist from the other party had “a large disciplining effect.”

  They found, for example, “strong evidence of ideological dampening” in environmental cases. Democrat appointees, who overall voted for plaintiffs 43% of the time, voted for plaintiffs just 10% of the time when sitting with two Republican appointees. Republican appointees, who overall voted for plaintiffs 20% of the time, voted for plaintiffs 42% of the time when seated with two Democrat appointees. This held up across most of the twenty-five categories of cases in which they had a sufficiently large sample to reach a conclusion.

  The authors concluded that the result endorsed the importance of exposure to diverse viewpoints: “What is necessary is reasonable diversity, or diversity of reasonable views . . . and that it is important to ensure that judges, no less than anyone else, are exposed to it, and not merely through the arguments of advocates.”

  Sunstein’s group found that federal appellate judges need the diverse viewpoint of an opposing-party appointee. Judges, they found, followed the human instinct of succumbing to groupthink. “Our data provide strong evidence that like-minded judges also go to extremes: the probability that a judge will vote in one or another direction is greatly increased by the presence of judges appointed by the president of the same political party. In short, we claim to show both strong conformity effects and group polarization within federal courts of appeals.”

  The growing polarization of the Supreme Court is a case in point. Each justice now has four clerks, all of whom have similar credentials: top-of-the-class graduates of top law schools, law review editors, and clerkships with federal appeals court judges. The clerks, over the years, have played an increasingly important role in helping the justices with their intellectual workload, discussing details of cases and drafting initial versions of opinions.

  Prior to the appointment of Chief Justice Roberts in 2005, it was an informal badge of honor, especially among some of the conservative members of the court, that they hired clerks with ideological backgrounds that differed from theirs. Bob Woodward and Scott Armstrong, in The Brethren, described how Justice Powell “prided himself on hiring liberal clerks. He would tell his clerks that the conservative side of the issues came to him naturally. Their job was to present the other side, to challenge him. He would rather encounter a compelling argument for another position in the privacy of his own chambers, than to meet it unexpectedly at conference or in a dissent.”

  Chief Justice Burger hired equally from the ranks of former clerks of Democrat- and Republican-appointed judges. Chief Justice Rehnquist, who served on the court with Burger and succeeded him, arrived at the court suspicious of the role liberal clerks could have in influencing his opinion. According to The Brethren, however, that attitude disappeared almost immediately. Rehnquist believed “the legal and moral interchanges that liberal clerks thrived on were good for the Justices and for the Court.” Justice Scalia, when he served on the D.C. circuit and in his early years on the Supreme Court, was known for seeking out clerks with liberal ideologies.

  As the Supreme Court has become more divided, this practice has all but ceased. According to a New York Times article in 2010, only Justice Breyer regularly employed clerks who had worked for circuit judges appointed by presidents of both parties. Since 2005, Scalia had hired no clerks with experience working for Democrat-appointed judges. In light of the shift in hiring practices, it should not be so surprising that the court has become more polarized. The justices are in the process of creating their own echo chambers.

  Justice Thomas, from 1986 to the time the article was written, was 84-for-84 in hiring clerks who had worked for Republican-appointed judges. Not surprisingly, according to data compiled from the Journal of Law, Economics, and Organization, he is the justice furthest from the ideological center of the court, much further right than the most liberal-leaning justice (Sotomayor) is left.

  Thomas once said, “I won’t hire clerks who have profound disagreements with me. It’s like trying to train a pig. It wastes your time, and it aggravates the pig.”* That makes sense only if you believe the goal of a decision group is to train people to agree with you. But if your goal is to develop the best decision process, that is an odd sentiment indeed.

  This polarization warns against forming a decision group that is a collection of clones who share the same opinions and knowledge sources we do. The more homogeneous we get, the more the group will promote and amplify confirmatory thought. Sadly, that’s exactly what we drift toward. Even Supreme Court justices do that. We are all familiar with this tendency in politics; it’s the complaint on both sides of the political aisle. Conservatives complain that liberals live in an echo chamber where they just repeat and confirm their point of view. They aren’t open to new information or ideas that don’t fit what they already believe. That’s the exact same criticism liberals have of conservatives.

  Although the Internet and the breadth of multimedia news outlets provide us with li
mitless access to diverse opinions, they also give us an unprecedented opportunity to descend into a bubble, getting our information from sources we know will share our view of the world. We often don’t even realize when we are in the echo chamber ourselves, because we’re so in love with our own ideas that it all just sounds sensible and right. In political discourse, virtually everyone, even those familiar with groupthink, will assert, “I’m in the rational group exchanging ideas and thinking these things through. The people on the other side, though, are in an echo chamber.”

  We must be vigilant about this drift in our groups and be prepared to fight it. Whether it is the forming of a group of friends or a pod at work—or hiring for diversity of viewpoint and tolerance for dissent when you are able to guide an enterprise’s culture toward accuracy—we should guard against gravitating toward clones of ourselves. We should also recognize that it’s really hard: the norm is toward homogeneity; we’re all guilty of it; and we don’t even notice that we’re doing it.

  Social psychologists: confirmatory drift and Heterodox Academy

  In 2011, Jon Haidt, speaking to an audience of 1,000 social psychologists, noted the lack of viewpoint diversity in their field. He reported that he could identify only one conservative social psychologist with any degree of field-wide recognition.

  Surveys of sociologists’ professional organizations have found that 85%–96% of members responding self-identified as left of center, voted for Obama in 2012, or scored left of center on a questionnaire of political views. (Most of the remaining 4%–15% identified as centrist or moderate rather than conservative.) The trend has a long tail, but it has been accelerating. In the 1990s, liberals among social psychologists outnumbered conservatives 4-to-1. More recent surveys show that the ratio has grown to greater than 10-to-1, sometimes far greater. A tendency to hire for a conforming worldview combined with the discouraging aspects of being so decisively outnumbered ideologically suggests that, unchecked, this situation won’t get better. According to the surveys establishing this trend toward homogeneity, about 10% of faculty respondents identified as conservative, compared with just 2% of grad students and postdoctoral candidates.