The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us

by Christopher Chabris

  These questions about the proper interpretation of the original Green-Bavelier study will be moot unless it can be consistently and independently replicated. One large-scale attempt to do just that, led by video-game researcher Walter Boot, did not produce the same results as the earlier experiments.68 Dan was one of the coauthors of Boot’s paper and participated in the design of the study. The original study and the replication by Feng’s group were both relatively small in scope: In each case, no more than ten subjects were assigned to each condition, and their training lasted only about ten hours. Boot’s study used more than twice as many subjects in each condition and gave the subjects more than twice as much training, over twenty hours on each game. He also used a much larger battery of cognitive tasks, including all of the ones used by Green and Bavelier plus about twenty others. The battery itself took up to two hours to complete, and each participant completed all the tasks before and after the training as well as once about halfway through it. Boot used the same Tetris and Medal of Honor games used in the original study, as well as the Rise of Nations game used in Basak’s experiment. Like Basak, he had the idea that training with that sort of strategy game would not enhance attention and perception, but instead would improve performance on measures of problem solving, reasoning, and possibly memory. Boot also included a group that received no training at all in order to provide a clear estimate of how much people might improve just by retaking the cognitive tasks before and after training. So this study was designed to test all of the alternative explanations for the positive findings that the original studies did not address—as well as the possibility that training released untapped potential.

  One oddity in all of the previous experiments showing positive evidence of video-game training is that none of the control groups did any better the second time they took the cognitive tests than they did the first time. In the original study by Green and Bavelier, the subjects who played Tetris (a video game, but not a fast-paced, first-person “action” game) showed no improvement when they did the cognitive tasks for a second time, after completing their training. The same was true for the replication by Feng and colleagues: Subjects in the control condition did no better when retaking the cognitive tasks. It also held true for most of the positive effects in the Basak study and for the subsequent studies conducted by Bavelier and her colleagues. Given what we know about practice and learning, this finding is hard to explain; people almost always perform better when they do a task a second time. Such improvements are typical as well for the sorts of tasks used in the Brain Age software and other brain-training products. In fact, these routine practice effects are exactly the “evidence” those programs rely on to back their claims that their users’ brains are “improving.”

  Why does lack of improvement in the control conditions matter? Because the evidence for the positive effects of video-game training is based on a comparison to these control groups. To support the claim that video games improve cognition, an experiment must show that people trained with video games improve more than people receiving other training or no training. It’s much easier to show an improvement relative to a control group if the control group shows no improvement at all. Had subjects in the control groups improved as expected, the benefits that could be ascribed to video games would have been reduced.

  In Boot’s experiment, unlike the others, the control group did show a typical increase in performance from the first to the last testing session. The group that practiced action video games also improved on the cognitive tasks. But it improved by the same amount as the control group, meaning that there was no specific effect of video-game training on cognitive abilities.69 This failure to replicate is especially significant because Boot doubled the amount of training and used more subjects and control groups—all of which strengthened the design of the study and made it a more definitive test of the broad transfer hypothesis advanced by Green and Bavelier. Their initially promising idea that a small amount of video-game training could have big effects does not seem to be borne out. It’s possible that some subtle differences in the methods among the various studies account for the different results, but if the effect is that fragile, it is hard to imagine that video games will turn out to be a panacea for cognitive decline.70

  Recall that the first four experiments in Green and Bavelier’s Nature article showed that video-game experts consistently outperformed novices on the same tasks that benefited from training in their experiment. Since the effects of training appear to be somewhat tenuous, you might now wonder why experts should tend to outperform novices. One explanation is that the cognitive differences between experts and novices might require a lot more than ten or even fifty hours of training to develop. The experts in these studies often play more than twenty hours of video games in a single week! If it takes that much effort to transfer skill from video games to general perception, would video-game training really be a worthwhile thing to do (if you didn’t already love playing video games)? The benefit of being a little faster on a selective attention task is probably not worth the hundreds of hours you would have to spend to receive it—you would be better off practicing the specific skills you are trying to improve. Given the lack of direct evidence that video-game training would even have consequences for our daily lives—say, by making us safer drivers—the potential benefits of training are even more uncertain.

  A more subtle concern is that the experts might not actually be any better at these cognitive tasks even if they do show better performance in the lab. How could that be? Some other factor unrelated to cognitive abilities might enhance performance. In his interview with Dan, Walter Boot raised a possibility rarely discussed in the scientific literature:

  Video-game experts might perform better because they know they have been selected to be in the study based on their expertise. Participants recruited through advertisements or flyers targeting gamers know they’re being selected because they’re an expert, because they are special, and they might be more motivated, more attentive, and have expectations that they should perform well. Because of all the media coverage, especially in blogs frequented by gamers, they know that they are expected to do better. And the nonexperts might not even know they are in a video-game study.71

  In other words, the experts might outperform the novices not because they are inherently better at these tasks or because they have thousands of hours of video-game experience, but because they know that the study is about video-game expertise and that they are expected to do better. This sort of “expectancy effect” is a well-known issue in this kind of experiment. One way to address the problem would be to recruit subjects without any mention of video games and then measure video-game expertise only after subjects are finished with all the cognitive tasks. That way, subjects would have no way of knowing that the study is about video-game expertise. Unfortunately, it’s an inefficient way to conduct a study, because you might need to test many additional subjects in order to have enough who meet the criteria for a novice or expert.

  Regardless of how the subjects are recruited, it is dangerous to draw any causal conclusions about the role of video games in cognition from studies of differences between expert and novice players—training experiments are essential to draw proper inferences about cause.72 Watch out for misreporting of such expertise effects in the media—journalists regularly claim that video games cause improvements when the studies they describe show only a difference between expert and novice players. Some writers have promoted the idea that video games have benefits extending far beyond increased attention or perceptual abilities—enhancing general intelligence, social ability, confidence, and logical thinking—with even less actual evidence for these claims.73

  Give Your Brain a Real Workout

  In promoting Brain Age, Nintendo’s website makes the following broad claim about how its products enhance brain function:

  Everyone knows you can prevent muscle loss with exercise, and use such activities to improve your body over time. And
the same could be said for your brain. The design of Brain Age is based on the premise that cognitive exercise can improve blood flow to the brain. All it takes is as little as a few minutes of play time a day. For everyone who spends all their play time at the gym working out the major muscle groups, don’t forget—your brain is like a muscle, too. And it craves exercise.74

  As it turns out, the final sentence is accurate, but not in the way that Nintendo’s marketers intended. They meant to imply that cognitive exercise is necessary to keep your brain functioning well. In reality, aerobic physical exercise is likely far better for your brain.75 Cognitive neuroscientist Arthur Kramer, a colleague of Dan’s at the University of Illinois, led one of the best-known studies of how improving physical fitness can affect cognitive abilities.76 Their experiment, published in Nature, randomly assigned 124 sedentary but otherwise healthy seniors to one of two training conditions for six months: aerobic fitness, in which the subjects spent about three hours each week walking, and an anaerobic exercise condition, in which subjects spent the same amount of time doing stretching and toning exercises. Although both forms of exercise are good for your body and lead to better overall fitness, aerobic exercise more effectively improves the health of your heart and increases blood flow to your brain.

  Not surprisingly, both training groups experienced the expected benefits to their physical fitness. The surprising result, though, is that walking for as little as a few hours a week also led to large improvements on cognitive tasks, particularly those that rely on executive functions like planning and multitasking. The stretching and toning exercise had no cognitive benefits. Kramer’s group also conducted a meta-analysis of all the clinical trials of the effects of aerobic fitness training on cognition through 2001; the results confirmed a sizable benefit of this type of fitness training for cognition.77

  The benefits of exercise are deeper than improvements in behavior and cognition. With age, most adults start to lose some of the gray matter in their brains. (This could be part of the reason for the accompanying cognitive declines.) In another clinical trial, Kramer’s group randomly assigned seniors to the same aerobic and anaerobic six-month training regimens just described, except this time, they first used MRI scanning to acquire a complete picture of each subject’s brain before and after the fitness training.78 The result was astounding: Seniors who had walked for just forty-five minutes a day for three days each week preserved much more gray matter in their frontal brain regions than did those who had done stretching and toning. Aerobic exercise actually did keep their brains healthier and younger.

  It might seem counterintuitive, but the best thing you can do to preserve and maintain your mental abilities may have little to do with cognition at all. Training your brain directly might have less impact than exercising your body, particularly if you exercise in a way that maintains your aerobic fitness. The exercise doesn’t even need to be particularly strenuous. You don’t need to compete in triathlons; just walking at a reasonable clip for thirty minutes or more a few times a week leads to better executive functioning and a healthier brain. Despite Nintendo’s claims that you need to exercise your brain, it seems that sitting in a chair and doing cognitive puzzles is far less beneficial than walking around the block a few times. Exercise improves cognition broadly by increasing the fitness of your brain itself. And doing puzzles does nothing for your longevity, your health, or your looks.

  conclusion

  the myth of intuition

  WHAT DO YOU LEARN WHEN you read profiles of corporate CEOs? You expect to find out what makes them tick: how they got to their current position, what inspired them to make the decisions they did, why their management style sets them up for success. And most important, you expect to learn about someone whose approach to business—and perhaps life in general—is worth emulating.

  As we discussed in Chapter 4, the only way to be sure that you understand something is to test your knowledge. Let’s do that now. Apply what you’ve learned about everyday illusions to this profile of business leader Larry Taylor. Some of the illusions will shine through, but others will be more subtle. See if you can spot them all.

  Larry Taylor is on his way to work. A stocky man with a military-style buzz cut and intense blue eyes, he sits ramrod straight behind the steering wheel. Despite being the CEO of Chimera Information Systems, a privately held corporation with more than $900 million in annual sales, he doesn’t have a driver. It would be awkward to have a driver when your car is just a Toyota Camry with cloth seats, not a Mercedes or Lexus with full leather and burl-wood interior. Taylor makes the forty-minute commute every day. En route, he talks with several of his top managers by phone, getting updates on software development projects, marketing plans, and sales progress—all before he arrives at the office.

  All you need to do is follow Taylor around for a few hours to see why his company’s revenues are growing at a rate of 45 percent per year, and why he was voted the most innovative and effective executive in the Midwest last year. According to industry analysts, Taylor’s arrival in the corner office in 2003 is the reason why Chimera has changed from a dowdy vendor of inventory-management software to an industry-leading developer of “middleware” for Web 2.0—applications that sit between a company’s public website and private data warehouses, managing communication between the two. Taylor’s next move will be to create software that enables even the smallest Internet retailers—the hundreds of thousands of EdsArgyleSocks.coms and eBay storefronts of the world—to manage their supply chains with the sophistication of an Amazon or a Walmart. According to Taylor, this is a $2 billion market opportunity that is wide open.

  Today, Taylor is talking to his chief financial officer, Jane Flynt, about Chimera’s quarterly earnings release that’s due in a week. Taylor speaks with the slight Texas drawl he acquired growing up in San Antonio. There is a pause in the conversation when Flynt steps away from her phone to ask an assistant to run some new analyses that Taylor suggested. At this moment, Taylor mutes his phone and explains the real reason he hired Flynt, who had never been the head of finance at a large company, over other candidates with Ivy League pedigrees and much more experience.

  “It was almost two years ago, but I remember it like it was yesterday,” says Taylor. “It was a crazy time … we needed to have a new CFO in place for the next board meeting, which was coming up fast, but I was traveling to see customers most days of the week back then. So I had them come in on a Sunday morning.” The four candidates on the short list duly showed up at 9:00 a.m., in their Sunday best. As a final “test” in the interview, Taylor handed out laptops with PowerPoint installed and asked each candidate to prepare and deliver a five-minute presentation on why he or she should be chosen as Chimera’s new CFO. And he told them that they had to deliver their presentations to him and to the other candidates, in the company boardroom. “When I said that, their jaws all dropped at once,” Taylor recalled. “They all had to be as nervous as a bunch of cats in a room full of rocking chairs.” He gave them just ten minutes with the computers to make their slides. “I picked Flynt to go first, and I thought she would wet herself. But she didn’t. She gave one of the best speeches I had ever heard in my life. What I kept thinking about was how self-assured she was under all the pressure of the situation I’d set up. I let the other guys give their talks, but I knew right then that I wanted Jane, and when the interviews were over I hired her on the spot.”

  Taylor is renowned at Chimera for the quickness with which he grasps complex ideas and information. “I only need to read a document once, and I pretty much completely understand it, and I’ll remember all of the details, too,” he tells us. A recent profile of Taylor in Inventory World reported that “Taylor says that he knows everything about how Chimera’s products work, often more than their own developers, whom he sometimes embarrasses with tough questions about software architecture and standards.”

  He is a voracious reader—not just of company reports, trade journals, and business b
ooks, but also the latest science and history, and even an occasional vampire novel to keep up with the current obsession of his teenage daughters. From his business and science reading, he’s picked up dozens of ideas that he’s implemented at Chimera. To boost the inventiveness and productivity of his software engineers, he ordered their managers to play classical music on the public address system for thirty minutes every day; behind the music, subliminal messages exhort employees to do their best.

  Taylor learned how to play poker in high school, and he showed a talent for it in college, quickly becoming the biggest winner at his fraternity’s regular game. After graduating, he spent a couple of years as a professional poker player on the tournament and cash-game circuit. Nowadays he finds his high-stakes action in the boardroom rather than the casino, but he still plays poker occasionally on the Internet, using the screen name “royalflushCEO.” Does his experience in poker influence his approach to business strategy? Is making a huge bluff to convince an opponent to fold a good hand the equivalent of making a risky but potentially rewarding investment in an unproven technology or market? “It doesn’t work like that,” Taylor says. “When I’m making a big decision for Chimera I don’t think about poker tactics. I think more about the broader lessons I took from the game. There’s a saying in poker that goes ‘think long, think wrong.’ It means that sometimes, the more you think about a decision, the more likely you are to make the wrong choice. I read Malcolm Gladwell’s book Blink, and it taught me that you have to go with your gut instincts, trust your intuition, when you’re faced with a complex, important decision.”