A Beautiful Math

by Tom Siegfried

  Montague and Berns's paper in that issue argued that the chemical dopamine was the brain's currency for gauging the relative payoffs of potential behaviors. The paper noted various lines of evidence supporting the idea that a circuit of activity linking two parts of the brain—one at the front, behind the forehead, and another deep in the brain's middle—helps govern choice making by producing more or less dopamine. Dopamine levels predict the likely reward associated with different choices, the evidence indicated.

  Dopamine had long been known as the brain's chief pleasure molecule, linked to behavior that produces pleasant feelings. But it's not merely pleasure that drives dopamine production. Actually, the brain's dopamine currency seems tuned to the expectation of pleasure (or reward of some sort). Some of the brain's dopamine-producing nerve cells are programmed to monitor the difference between expected and actual reward, Montague and Berns showed. If a choice produces precisely the predicted reward, the dopamine cells maintain a constant level of activity. When pleasure exceeds expectations, the cells squirt out dopamine like crazy. If the reward disappoints, dopamine production is curtailed. This monitoring system also takes timing into account—if dinner is delayed, dopamine is diminished. When the anticipated rewards aren't realized, the dopamine monitoring system tells the brain to change its behavior. In this way the expectation of reward can guide a brain's decisions.

  A critical point, noted by Montague and Berns, is that all brains are not alike. One person's dream reward might be another's horrific nightmare. Some people make a risky choice only when expecting a huge reward; others gamble for the fun of it. Part of the promise of neuroeconomics is its ability to identify such individual differences with brain scanning.

  In one experiment described by Montague and Berns, people chose either A or B on a computer screen and then watched a bar on the screen to see whether their choice earned a reward. (The bar recorded accumulated reward "points" as the game progressed.) As the game went on, the computer adjusted the rewards, based on the player's choices. At first, choosing A raised the bar more, but choosing A too often made B a better bet. When A's payoffs dropped, some players noticed right away and quickly switched to choosing B more often. But others stuck with A, gambling that it would return to its previous high-payoff rate. It appeared that some brains are more inclined to take risks than others—some players play conservatively; others are risk-takers. (Actually, Montague said, more accurate labels for the two types of players would be "matchers" and "optimizers." "I call them conservative and risky because you can make good jokes about that," he said.)

  To me, it sounds more like they should be called "switchers" and "stickers." But the labels don't really matter. The most intriguing result from this experiment is the revelations from the brain scans. Sure enough, patterns of brain activity differed in the two groups, particularly in a small clump of brain cells called the nucleus accumbens. It's a brain region implicated in drug addiction, and it's more active in the "risk-taking" game players (the stickers).

  The neatest thing, though, is that you can tell who the risk takers and play-it-safers are from their brain scans just after the very beginning of the game, even while their behaviors are still identical. This is the sort of evidence that destroys the old behaviorist position that behavior is the only thing that matters (or that you can know). Early in the game, two players can behave identically, making exactly the same choices. Yet by looking into their brains you can see differences that allow you to predict how they will play later, when the payoff rate changes.

  "The people that ended up on average being risky are different from these people right away—nobody even jumps categories," Montague told me. Even more intriguing, there appears to be a genetic difference between the two groups as well.

  So neuroeconomics thus offers economists a tool they had not possessed before, giving hope that by getting inside people's heads, science might really be on the road to finding the Code of Nature that governs human behavior.

  WHOM DO YOU TRUST?

  An important advance along that road came in 2003 with the publication of a paper in the journal Science by researchers at Princeton University. In a study by Alan Sanfey and colleagues, participants in an experiment played the ultimatum game, one of the favorites of behavioral game theorists. It's kind of like a TV game show contest in which you are given a lot of money, but you have to share your windfall with a stranger. Suppose you get $100. You then offer the stranger part of the money and keep the rest— unless the stranger refuses your offer. Then you have to give all the money back, and nobody wins anything.

  In theory, the stranger should take any offer, no matter how small, in order to get something rather than nothing. Therefore, a game theorist might conclude, you should offer a low amount— $10, say, or even $1—so that you will then walk away with the most money possible. But in practice, most strangers reject low offers. If you offer $10, for instance, you're much more likely to walk away with zero than $90, as the stranger will probably reject your offer just to punish you, even at personal expense. Consequently people typically share more generously—offering 40 to 50 percent of the prize, say—in anticipation of an angry rejection of an unfair offer.

  So this is another case where naive game theory, in assuming that everybody will maximize their money, makes an incorrect prediction, as many economic experiments had already established. The Princeton study went further, though, by scanning the brains of the strangers who were considering whether to accept the offer from the other participant. In this case, the prize was only $10— science doesn't have budgets like Who Wants to Be a Millionaire?— but the principle was the same. If the first player offered only $1 or $2, the offer was usually rejected. But not always. And you could tell who was likely to accept or reject a low offer by watching what went on inside their brains.

  Stronger brain activity in the front part of a brain region known as the insula (an area known to be associated with negative emotions, such as anger and disgust) was common in players who were more likely to reject low offers. Another brain structure—the anterior cingulate cortex—also showed increased activity in those who rejected unfair offers. That region is known to be involved in monitoring conflict—in this case, the conflict between the choice of punishing a cheapskate or turning away money. "Unfair treatment … can lead people to sacrifice sometimes considerable financial gain in order to punish their partner for the slight," Sanfey and his collaborators reported in Science.10

  In a commentary on that paper, Colin Camerer noted that it showed how the tenets of basic game theory do not always hold— people do not always act totally in their own self-interest (that is, maximizing their money), and all the players in a "game" therefore are not always trying to do the best they can do, as assumed in the underlying basis for a Nash equilibrium. But behavioral game theory, Camerer noted, can relax these assumptions and still learn a lot about human behavior. The neuroeconomics enterprise, in other words, is a powerful tool for developing behavioral game theory insights into how real people make choices.

  Montague's subjects at Baylor, for instance, play similar behavioral games that reveal the quirks of human economic behavior. In one such game—a task for testing trust—Player 1 is given $20 and is allowed to keep some of it and put the rest in a virtual pot, where the amount is then tripled. If Player 1 keeps $10 and donates $10, the sum in the pot becomes $30. Player 2 then gets to split the pot with Player 1—or take it all.

  "If you split it 15-15, then in a sense you've repaid the trust," said Montague. But if you take $29 and leave $1, Player 1 is not likely to offer much in the next round of the game. At any point in the game, one player or the other could decide to keep all the money, so the logical move is to take it all as soon as possible, before the other player does. But in fact, players typically trust each other not to be so selfish—although some are more trusting, and some more selfish, than others.

  Traditional economists were not surprised at the results of such games. In th
e 1980s, game theory had fueled the rise of "experimental economics" in which such deviations from pure self-interest showed up regularly. What's new in neuroeconomics is eavesdropping on the players' brains via the MRI scanners while the games are in progress. Montague's lab is particularly well equipped for this sort of thing, with a pair of scanners, one each in two rooms separated by the scientists' observing station. The scientists watch as computers record the brain activity of players deciding how to move or how to react to another player's move. "You can see what went on in the behavior. You can back up and look at their intent to act badly or their intent to invest more," Montague said. "It allows us to cross-correlate what's going on in the two brains. I think it's cool. I think it's an obvious way to study social interactions."11

  Neuroeconomics does not always require scanning, though. Paul Zak, director of the Center for Neuroeconomics Studies at Claremont Graduate University in California, sometimes uses blood tests instead of brain scans. He can relate variant economic behaviors to levels of certain hormones. In one of Zak's versions of the trust game, players communicate via computer. One player, given $10, offers some of it to another player, who is paid triple the amount offered. (So if Player 1 offers $5, Player 2 gets $15). Player 2 then can take it all, or give part of it back to Player 1. But in this version of the experiment, the game ends after just one round. There's no incentive to earn trust so as to get more money the next time around.

  So standard game theory suggests that Player 2 would take all the money, having nothing to gain by giving some back. But Player 1, anticipating that move, should therefore offer none of the money to begin with. Nevertheless, many players defy naive game theory and show at least some trust that the other player will play fair. About half of the first-movers offer some money (suggesting that they are trusting souls), while three in four of the responders give some back (suggesting that they are trustworthy).

  Once again, the intriguing thing about such games is finding out what's behind the differences in individual behavior. It turns out that among the trustworthy players, blood tests revealed higher levels of oxytocin, a hormone linked to pleasure and happiness. Apparently the trusting gesture of the first player, by offering some money, elicits a positive hormonal response. "It tells us that people are very much responsive to their environment," Zak told me when I visited him at Claremont. "People who got a positive signal had a nice positive happy hormone response, and their behavior reflects that."12

  Zak believes that the relationship between trust and oxytocin is central to understanding many of the world's economic ills. Oxytocin is linked to happiness, and the countries where people report high levels of happiness are also countries where people report high degrees of trust. Trust levels, in turn, are a good indicator of a country's economic well-being. "Trust is among the biggest things economists have ever found that are related to economic growth," Zak said.

  HOMO NEUROECONOMICUS

  For all of its intriguing findings, neuroeconomics doesn't excite everybody, like the economist who perplexed Montague by not caring about the brain. From the perspective of economists like that one, neuroeconomics probably doesn't have much to offer. To them, it only matters what people do; it doesn't matter which part of the brain is busy when they do it.

  Neuoreconomists, though, want more than a mere description of economic decision making. They want the Code of Nature, the scientific understanding of humanity sought by 18th-century thinkers such as David Hume and Adam Smith. "The more ambitious aim of neuroeconomics," writes neuroeconomist Aldo Rustichini, "is going to be the attempt to complete the research program that the early classics (in particular Hume and Smith) set out in the first place: to provide a unified theory of human behavior."13

  Rustichini, of the University of Minnesota, points out that Adam Smith's great works—Theory of Moral Sentiments and Wealth of Nations—were part of a grand plan to codify the nature of human civilization, to explain how selfish individuals manage to cooperate sufficiently well to establish elaborate functioning societies. Smith's basic answer was the existence of sympathy—the ability of one human to understand what another is feeling. Modern neuroscience has begun to show how sympathy works, by identifying "mirror neurons," nerve cells in the brain that fire their signals both in performing an action and when viewing someone else performing that same action.

  Other neuroscientific studies have identified the neural basis of both individual behavioral propensities and collective and cooperative human behavior. Scientists scanning the brains of players participating in a repeated Prisoner's Dilemma game, for instance, have identified regions in the brain that are active in players who prefer cooperating rather than the "purely rational" choice to defect.14

  Another study used a version of the trust game to examine the brains of people who punish those who play uncooperatively (by keeping all the money instead of returning a fair share). In this game, players who feel cheated may assess a fine on the defector (even though they must pay the price of reducing their own earnings by half the amount of the fine they impose). People who choose to fine the defector display extra activity in a brain region associated with the expectation of reward. That suggests that some people derive pleasure from punishing wrongdoers—the payoff is in personal satisfaction, not in money. In the early evolution of human society, such "punishers" would serve a useful purpose to the group by helping to ostracize the untrustworthy noncooperators, making life easier for the cooperators. (Since this punishment is costly to the individual but beneficial to the group as a whole, it is known as "altruistic punishment.")15

  Such studies highlight an essential aspect of human behavior that a universal Code of Nature must accommodate—namely that people do not all behave alike. Some players prefer to cooperate while others choose to defect, and some players show a stronger desire than others to inflict punishment. A Code of Nature must accommodate a mixture of individually different behavioral tendencies. The human race plays a mixed strategy in the game of life. People are not molecules, all alike and behaving differently only because of random interactions. People just differ, dancing to their own personal drummer. The merger of economic game theory with neuroscience promises more precise understanding of those individual differences and how they contribute to the totality of human social interactions. It's understanding those differences, Camerer says, that will make such a break with old schools of economic thought.

  "A lot of economic theory uses what is called the representative agent model," Camerer told me. In an economy with millions of people, everybody is clearly not going to be completely alike in behavior. Maybe 10 percent will be of some type, 14 percent another type, 6 percent something else. A real mix.

  "It's often really hard, mathematically, to add all that up," he said. "It's much easier to say that there's one kind of person and there's a million of them. And you can add things up rather easily." So for the sake of computational simplicity, economists would operate as though the world was populated by millions of one generic type of person, using assumptions about how that generic person would behave.

  "It's not that we don't think people are different—of course they are, but that wasn't the focus of analysis," Camerer said. "It was, well, let's just stick to one type of person. But I think the brain evidence, as well as genetics, is just going to force us to think about individual differences."

  And in a way, that is a very natural thing for economists to want to do.

  "One of the most central and interesting things in economics is specialization and division of labor," Camerer observed. "And so loosely speaking, the more individual difference there is, the better that might be for the economy—as long as you get people in the right jobs. And so knowing more about individual differences could be very important for areas like labor economics, where one of the central questions is are you matching the right workers to the right jobs."16

  Zak, who has also performed studies to localize the brain's computing of utility, notes that such work rev
olutionizes the kinds of questions that economists can study.

  "In economics we generally think of this utility function as pretty much uniform across individuals," he said. "Now we can ask all kinds of questions about that. How stable is it, how different is it across people, why do you prefer coffee and I prefer tea? What if the price of coffee went up twice as much, what if you haven't drunk coffee in two weeks? Do you value it more, do you value it less? These are really basic questions that may affect things like how things are priced in the market and it may affect how we design laws."17

  Yet while neuroeconomics may provide the foundation for understanding individual behavior and differences, it cannot alone provide the Code of Nature, or a science of human behavior like Asimov's psychohistory. History comprises the totality of collective human behavior in various forms of social interaction— politically, economically, and culturally. It's in understanding human culture that science must seek a Code of Nature, and game theory provides the best tool for that task.

  * * *

  6

  Seldon's Solution Game theory, culture, and human nature

  Self-interest speaks all sorts of languages and plays all sorts of roles.

  —La Rochefoucauld

  You don't need to know about game theory to understand the ultimatum game. You just need to be a movie fan.

  Decades before economists invented the ultimatum game,1 something very much like it appeared in the 1941 movie The Maltese Falcon. The scene is private detective Sam Spade's apartment. Spade (played by Humphrey Bogart) has just made a deal with the criminal Kasper Gutman (Sydney Greenstreet). Spade will collect $1,000 from Gutman and then presumably will share some of it with Brigid O'Shaughnessy (Mary Astor), the film's femme fatale.