The Intelligence Trap

by David Robson

  These are highly intimate problems, but taking a distant viewpoint also seems to remedy bias on less personal subjects. When told to imagine how citizens in other countries would view forthcoming elections, for instance, Grossmann’s participants became more open-minded to conflicting views. After the experiment, he found that they were also more likely to take up an offer to sign up for a bipartisan discussion group – offering further, objective evidence that they were now more open to dialogue as a result of the intervention.46

  As the research evolves, Grossmann has now started to examine the conditions of the effect more carefully, so that he can find even more effective self-distancing techniques to improve people’s reasoning. One particularly potent method involves imagining that you are explaining the issue to a twelve-year-old child. Grossmann speculates that this may prime you to be more protective, so that you avoid any bias that could sway their young and naïve mind.47

  His team call this phenomenon the ‘Socrates Effect’ – the humble, Greek philosopher correcting the egocentric passions of the mighty Israelite king.

  If you still doubt that these principles will help you make better decisions, consider the achievements of Michael Story, a ‘super-forecaster’ whose talents first came to light through the Good Judgment Project – a US government-funded initiative to improve its intelligence programme.

  The Good Judgment Project was the brainchild of Philip Tetlock, a political scientist who had already caused shockwaves among intelligence analysts. Whenever we turn on the TV news or read a newspaper, we meet commentators who claim to know who will win an election or if a terrorist attack is imminent; behind closed doors, intelligence analysts may advise governments to go to war, direct NGOs’ rescue efforts or advise banks on the next big merger. But Tetlock had previously shown that these professionals often perform no better than if they had been making random guesses – and many performed consistently worse.

  Later research has confirmed that their rapid, intuitive decision making makes many intelligence analysts more susceptible to biases such as framing – scoring worse than students on tests of rationality.48

  It was only after the US-led invasion of Iraq in 2003 – and the disastrous hunt for Saddam Hussein’s ‘Weapons of Mass Destruction’ – that the US intelligence services finally decided to take action. The result was the founding of a new department – Intelligence Advanced Research Projects Activity. They eventually agreed to fund a four-year tournament, beginning in 2011, allowing researchers to arrange the participants in various groups and test their strategies.

  Example questions included: ‘Will North Korea detonate a nuclear device before the end of the year?’ ‘Who will come top of the 2012 Olympics medals table?’ And, ‘How many additional countries will report cases of the Ebola virus in the next eight months?’ In addition to giving precise predictions on these kinds of events, the forecasters also had to declare their confidence in their judgements – and they would be judged extra harshly if they were overly optimistic (or pessimistic) about their predictions.

  Tetlock’s team was called the Good Judgment Project, and after the first year he siphoned off the top 2 per cent, whom he called the ‘super-forecasters’, to see if they might perform better in teams than by themselves.

  Michael joined the tournament midway through the second year, and he quickly rose to be one of the most successful. Having worked in various jobs, including documentary film-making, he had returned to academia for a Master’s degree, when he saw an advert for the tournament on an economics blog. The idea of being able to test and quantify his predictions instantly appealed.

  Michael can still remember meeting other ‘supers’ for the first time. ‘There are loads of weird little things about us that are very similar,’ he told me; they share an inquisitive, hungry mind with a thirst for detail and precision, and this was reflected in their life decisions. One of his friends compared it to the ending of ET, ‘where he goes back to his home planet, and he meets all the other ETs’.

  Their observations tally with Tetlock’s more formal investigations. Although the super-forecasters were all smart on measures of general intelligence, ‘they did not score off-the-charts high and most fall well short of so-called genius territory’, Tetlock noted. Instead, he found that their success depended on many other psychological traits – including the kind of open-minded thinking, and the acceptance of uncertainty, that was so important in Grossmann’s research. ‘It’s being willing to acknowledge that you have changed your mind many times before – and you’ll be willing to change your mind many times again,’ Michael told me. The super-forecasters were also highly precise with their declarations of confidence – specifying 22 per cent certainty, as opposed to 20 per cent, say – which perhaps reflects an overall focus on detail and precision.

  Tetlock had already seen signs of this in his earlier experiments, finding that the worst pundits tended to express themselves with the most confidence, while the best performers allowed more doubt to creep into their language, ‘sprinkling their speech with transition markers such as “however”, “but”, “although” and “on the other hand” ’.

  Remember Benjamin Franklin’s determination to avoid ‘certainly, undoubtedly, or any other [phrases] that give the air of positiveness to an opinion’? More than two hundred years later, the super-forecasters were again proving exactly the same point: it pays to admit the limits of your knowledge.

  In line with Grossmann’s research, the super-forecasters also tended to look for outside perspectives; rather than getting stuck in the fine details of the specific situation at hand, they would read widely and look for parallels with other (seemingly unconnected events). Someone investigating the Arab Spring, for instance, may look beyond Middle Eastern politics to see how similar revolutions had played out in South America.

  Interestingly, many of the super-forecasters – including Michael – had lived and worked abroad at some point in their life. Although this may have just been a coincidence, there is some good evidence that a deep engagement with other cultures can promote open-minded thinking, perhaps because it demands that you temporarily put aside your preconceptions and adopt new ways of thinking.49

  The most exciting result, however, was the fact that these skills improved with training. With regular feedback, many people saw their accuracy slowly climbing over the course of the tournament. The participants also responded to specific lessons. An hour-long online course to recognise cognitive bias, for instance, improved the forecasters’ estimates by around 10 per cent over the following year.

  Often, the simplest way to avoid bias was to start out with a ‘base rate’: examining the average length of time it takes for any dictator to fall from power, for instance – before you then begin to readjust the estimate. Another simple strategy was to examine the worst- and best-case scenarios for each situation, offering some boundaries for your estimates.

  Overall, the super-forecasters provided the perfect independent demonstration that wise decision making relies on many alternative thinking styles, besides those that are measured on standard measures of cognitive ability. As Tetlock puts it in his book Superforecasting: ‘A brilliant puzzle-solver may have the raw material for forecasting, but if he doesn’t also have an appetite for questioning basic, emotionally charged beliefs, he will often be at a disadvantage relative to a less intelligent person who has a greater capacity for self-critical thinking.’50

  Grossmann says that he has only just come to appreciate these parallels. ‘I think there is quite a bit of convergence in those ideas,’ he told me.

  Michael now works for a commercial spin-off, Good Judgment Inc., which offers courses in these principles, and he confirms that performance can improve with practice and feedback. However you perform, it’s important not to fear failure. ‘You learn by getting it wrong,’ Michael told me.

  Before I finished my conversation with Grossmann, we discussed one final, fascinating experiment that took his wise reasonin
g tests to Japan.

  As in Grossmann’s previous studies, the participants answered questions about news articles and agony aunt columns, and were then scored on the various aspects of wise reasoning, such as intellectual humility, the ability to take on board another viewpoint, and their ability to suggest a compromise.

  The participants ranged from twenty-five to seventy-five years old, and in the USA, wisdom grew steadily with age. That’s reassuring: the more we see of life, the more open-minded we become. And it’s in line with some of the other measures of reasoning, such as Bruine de Bruin’s ‘adult decision-making competence scale’, in which older people also tend to score better.

  But Grossmann was surprised to find that the scores from Tokyo took a completely different pattern. There was no steep increase in age, because the younger Japanese were already as wise as the oldest Americans. Somehow, by the age of twenty-five, they had already absorbed the life lessons that only come to the Americans after decades more experience.51

  Reinforcing Grossmann’s finding, Emmanuel Manuelo, Takashi Kusumi and colleagues recently surveyed students in the Japanese cities of Okinawa and Kyoto, and Auckland in New Zealand, on the kinds of thinking that they thought were most important at university. Although all three groups recognised the value of having an open-minded outlook, it’s striking that the Japanese students referred to some specific strategies that sound very much like self-distancing. One student from Kyoto emphasised the value of ‘thinking from a third person’s point of view’, for instance, while a participant in Okinawa said it was important to ‘think flexibly based on the opposite opinion’.52

  What could explain these cultural differences? We can only speculate, but many studies have suggested that a more holistic and interdependent view of the world may be embedded in Japanese culture; Japanese people are more likely to focus on the context and to consider the broader reasons for someone’s actions, and less likely to focus on the ‘self’.53

  Grossmann points to ethnographic evidence showing that children in Japan are taught to consider others’ perspectives and acknowledge their own weaknesses from a young age. ‘You just open an elementary school textbook and you see stories about these characters who are intellectually humble, who think of the meaning of life in interdependent terms.’

  Other scholars have argued that this outlook may also be encoded in the Japanese language itself. The anthropologist Robert J. Smith noted that the Japanese language demands that you encode people’s relative status in every sentence, while the language lacks ‘anything remotely resembling the personal pronoun’. Although there are many possible ways to refer to yourself, ‘none of the options is clearly dominant’, particularly among children. ‘With overwhelming frequency, they use no self-referent of any kind.’

  Even the pronunciation of your own name changes depending on the people with whom you are speaking. The result, Smith said, is that self-reference in Japan is ‘constantly shifting’ and ‘relational’ so that ‘there is no fixed centre from which the individual asserts a non-contingent existence’.54 Being forced to express your actions in this way may naturally promote a tendency for self-distancing.

  Grossmann has not yet applied his wise reasoning tests to other countries, but converging evidence would suggest that these differences should be considered part of broader geographical trends.

  Thanks, in part, to the practical difficulties inherent in conducting global studies, psychologists once focused almost entirely on Western populations, with the vast majority of findings emerging from US university students – highly intelligent, often middle-class people. But during the last ten years, they have begun to make a greater effort to compare the thinking, memory and perception of people across cultures. And they are finding that ‘Western, Educated, Industrialised, Rich, Democratic’ (WEIRD, for short) regions like North America and Europe score higher on various measures of individualism and the egocentric thinking that appears to lie behind our biases.

  In one of the simplest ‘implicit’ tests, researchers ask participants to draw a diagram of their social network, representing their family and friends and their relationships to each other. (You could try it for yourself, before you read on.)

  In WEIRD countries like the USA, people tend to represent themselves as bigger than their friends (by about 6 mm on average) while people from China or Japan tend to draw themselves as slightly smaller than the people around them.55 This is also reflected in the words they use to describe themselves: Westerners are more likely to describe their own personality traits and achievements, while East Asian people describe their position in the community. This less individualistic, more ‘holistic’ way of viewing the world around us can also be seen in India, the Middle East and South America,56 and there is some emerging evidence that people in more interdependent cultures find it easier to adopt different perspectives and absorb other people’s points of view – crucial elements of wisdom that would improve people’s thinking.57

  Consider measures of over-confidence, too. As we have seen, most WEIRD participants consistently over-estimate their abilities: 94 per cent of American professors rate themselves as ‘better than average’, for instance, and 99 per cent of car drivers think they are more competent than the average.58 Yet countless studies have struggled to find the same tendency in China, Korea, Singapore, Taiwan, Mexico or Chile.59 Of course, that’s not to say that everyone in these countries will always be humble, wise thinkers; it almost certainly depends on the context, as people naturally flip between different ways of thinking. And the general characteristics may be changing over time. According to one of Grossmann’s recent surveys, individualism is rising across the globe, even in populations that traditionally showed a more interdependent outlook.60

  Nevertheless, we should be ready to adopt the more realistic view of our own abilities that is common in East Asian and other cultures, as it could directly translate to a smaller ‘bias blind spot’, and better overall reasoning.

  We have now seen how certain dispositions – particularly intellectual humility and actively open-minded thinking – can help us to navigate our way around the intelligence trap. And with Franklin’s moral algebra and self-distancing, we have two solid techniques that can immediately improve our decision making. They aren’t a substitute for greater intelligence or education, but they help us to apply that brainpower in a less biased fashion, so that we can use it more fruitfully while avoiding any intellectual landmines.

  The science of evidence-based wisdom is still in its infancy, but over the next few chapters we will explore convergent research showing how cutting-edge theories of emotion and self-reflection can reveal further practical strategies to improve our decision making in high-stakes environments. We’ll also examine the ways that an open-minded, humble attitude, combined with sophisticated critical thinking skills, can protect us from forming dangerous false beliefs and from ‘fake news’.

  Benjamin Franklin continued to embody intellectual humility to the very end. The signing of the Constitution in 1787 was his final great act, and he remained content with his country’s progress. ‘We have had a most plentiful year for the fruits of the earth, and our people seem to be recovering fast from the extravagant and idle habits which the war had introduced, and to engage seriously in the contrary habits of temperance, frugality, and industry, which give the most pleasing prospects of future national felicity’, he wrote to an acquaintance in London in 1789.61

  In March 1790, the theologian Ezra Stiles probed Franklin about his own beliefs in God and his chances of an afterlife. He replied: ‘I have, with most of the Dissenters in England, some doubts as to [Jesus’s] divinity, though it is a question I do not dogmatise upon, having never studied it, and think it needless to busy myself with it now, when I expect soon an opportunity of knowing the truth with less trouble.

  ‘I shall only add, respecting myself, that, having experienced the goodness of that Being in conducting me prosperously through a long life, I have no
doubt of its continuance in the next, though without the smallest conceit of meriting such goodness.’62 He died little more than a month later.

  5

  Your emotional compass: The power of self-reflection

  As he hungrily ate his burger and fries, Ray had already begun to sketch out his business plan. The fifty-two-year-old salesman was not a gambling man, but when he got this intense visceral feeling in his ‘funny bone’, he knew he had to act – and he had never before felt an intuition this strong.

  Those instincts hadn’t led him astray yet. He had made his way from playing jazz piano in bars and bordellos to a successful career in the paper cup industry, becoming his company’s most successful salesman. Then, soon after the Second World War, he had seen the potential in milkshake mixers, and he was now making a tidy sum selling them to diners.

  But his mind was always open to new possibilities. ‘As long as you’re green you’re growing; as soon as you’re ripe you start to rot,’ he liked to say. And although his body may have been telling him otherwise – he had diabetes and the beginnings of arthritis – he still felt as green as people half his age.

  So when he noticed that new clients were flocking to him on the recommendation of one particular hamburger joint, owned by two brothers in San Bernadino, California, he knew he had to take a look. What was so special about this one outlet that had inspired so many others to pay out for a better shake maker?

  Entering the premises, he was struck first by the cleanliness of the operation: everyone was dressed in pristine uniforms, and unlike the typical roadside restaurant, it wasn’t swarming with flies. And although the menu was limited, the service was quick and efficient. Each step of the food production was stripped down to its essence, and by paying with the order, you could come in and go out without even having to wait around tipping waitresses. Then there was the taste of the French fries, cut from Idaho potatoes that were cooked to perfection in fresh oil, and the burgers, fried all the way through with a slice of cheese on one side. You could, the sign outside read, ‘buy ’em by the bag’.