It’s hard for anyone to enforce policies that day in and day out tick off colleagues. That’s especially true if these colleagues are the ones who choose which partners will get to be the senior managing partners. In partnerships like Arthur Andersen or any of the other big accounting firms (or law firms), the people who run the organization are elected by their colleagues. Their engagement partners, not the senior managers, are the rainmakers who keep money pouring in.
The perverse incentive structure that discourages companies from accurately anticipating fraud is not unique to the accounting business. We can see the same problems in the insurance and banking industries. Suppose, for instance, you told underwriters to stop selling directors’ and officers’ insurance to a big client like Enron in 1995. In 2001 the SEC alleged that Enron had committed securities fraud starting around 1997 or 1998. Before that, Enron was a well-regarded company. During all of those years between 1995 and 2001, your colleagues, the insurance underwriters, would be screaming that you were taking their income away, that there was no evidence that Enron was doing anything wrong, that in fact it was a fine and prosperous company. In their eyes, you were giving their business away to people working for rival firms. That’s a pretty tough case to refute while you wait five, ten, or fifteen years for the other shoe to drop. You can imagine how hard it must be to get a real commitment to monitor and punish misconduct, since one must be careful, of course, not to jump in and punish employees or clients before you are sure they have done something wrong. There are big costs attached to falsely accusing a client of fraud, just as there are big costs attached to incorrectly trusting that a firm is behaving honestly.
Management can be a profile in courage by cutting off revenues today to prevent bigger headaches tomorrow, but most profiles in courage, as it turns out, lose their jobs. That’s not an easy choice for anyone. Sure, we all pay lip service to the idea that we should do what is good for us and our colleagues in the long run, but doing what is good in the long run can be very costly in the near term. As Lord Keynes so aptly observed, in the long run we’re all dead (or, anyway, retired). Losing business now to avoid lawsuits later is hard for exactly that reason.
As we’ve explored, game theory predicts that people frequently, for rational reasons, assume great risk and experience great failure. I suppose you could say that making predictions for a living makes that very possibility something of a daily routine. Thankfully, my record has been pretty good, but there have been some notable misses. And indeed there are some other associated risks with the further refinement of rational choice theory and the models I develop and employ. The next chapter will examine some of these issues.
8
HOW TO PREDICT THE UNPREDICTABLE
THIS CHAPTER IS about the limitations of my models, some of my worst-ever predictions, and some of the potential dangers that can conceivably stem from “predictioneering.” Many a critic of mine will have well-worn and dog-eared pages in this stretch of the book!
My worst-ever prediction came in the months after Bill Clinton’s election to the presidency. When he was elected, it was obvious to everyone that he was going to try to push through a comprehensive health care plan. He assigned his wife to head a task force charged with designing a health program. At the time, I was engaged by a major brokerage firm to help work out what was likely to get through Congress so that they could design investment opportunities around the new program. As we all know now, the task force created a lot of heat, but no agreement on a new health care program. Instead, it failed dismally.
As it happened, my analysis of what the health care plan would look like led to one of my worst-ever predictions. Each and every detail of what came out of my analysis was both wrong and filled with lessons that improved future assessments. Models fail for three main reasons: the logic fails to capture what actually goes on in people’s heads when they make choices; the information going into the model is wrong—garbage in, garbage out; or something outside the frame of reference of the model occurs to alter the situation, throwing it off course. The last of these is what happened to my health care analysis.
In early 1993, I predicted what was likely to get through Congress sometime in 1993 or 1994. In some sense, all three of the limitations I mentioned were involved and were subsequently addressed as part of my personal learning experience. But by and large, the main problem had to do with an unforeseen event that completely altered the setting in which health care was going to be shepherded through Congress. Of course, the whole point of prediction is to forecast the unforeseen. Anyone can predict that the sun will rise in the east and set in the west tomorrow. Still, there’s unforeseen and then there’s unforeseen. I think you’ll see what I mean when we go through what happened to health care, at least as I looked at it.
Although the experts who provided the data identified a great many components of a comprehensive health plan—including questions related to long-term care, proportion of the population covered, costs of drugs, distribution of the tax burden for health care across the federal and state governments, as well as employers’ costs, total spending on health care, and even questions related to ancillary care—that would get congressional approval, none did. As it happens, the model predicted that Daniel Rostenkowski, then an influential Illinois congressman and, crucially, chairman of the powerful House Ways and Means Committee, was the key to getting health care legislation through Congress. Mr. Rostenkowski, however, was indicted on seventeen felony counts of corruption in 1994 (and later convicted) based on investigations that reached their height during 1993, as the Clinton White House’s health care push began in earnest. Rostenkowski’s salience for health care plummeted, of course, first in anticipation of his indictment and then even more as he fought to salvage his reputation, maintain his leadership position in Congress, and keep himself out of prison. He failed on all counts, and my prediction, based on his effective efforts on behalf of health care, also failed. As a result, contrary to my expectations, nothing passed through Congress.
Rostenkowski’s indictment was a shattering shock to the situation as analyzed; I’ll explain why in a moment. The model assumed, incorrectly, that the underlying conditions would remain unaltered during the period of negotiation and bargaining over health care. My client was not terribly happy that I got everything wrong, and neither was I, but at least I had the benefit of learning an important lesson. It was little consolation to know that if I repeated my analyses after dropping Rostenkowski from the data set, I got everything right. Without Rostenkowski, the model showed that agreement would not be reached in the House of Representatives, and that meant there would be no comprehensive health care plan. But, of course, that was analysis done in hindsight, and that is no way to help a client. Needless to say, the client was not particularly understanding or forgiving and never asked me to do another piece of work—a great disappointment, because I would have welcomed the opportunity to prove to them the value of modeling, and to do so for free. But they didn’t bite, and who can blame them? They invested valuable time as well as money in my analysis, and they had absolutely nothing to show for it.
What did my study find and why did it find it? The Rostenkowski study, as I now think of it, had a long list of players that included several members of Congress, Hillary Clinton herself, health care expert advisers from nursing homes, AARP, pharmaceutical companies, employers of all shapes and sizes, and so forth. Many issues were relatively difficult to resolve within the model’s own logic, taking many rounds of negotiation, posturing, and information exchanges before settling on what looked like a stable outcome—that is, an outcome that could get through the House and Senate. It was evident that more compromise was needed than some key players were prepared to accept. It was also evident that the study had to involve at least two (and possibly as many as four) distinct phases.
The first phase, common in many analyses of legislative decisions, focused on the period of lobbying and jockeying for position. In this phase,
all of the players with an interest in shaping the outcome are part of the analysis. That includes many stakeholders who would not have a place at the table when it came time for the House and Senate to vote and for the president to sign or veto whatever they sent up to him. Organizations like Blue Cross—Blue Shield or the AMA that were utterly opposed to the Clinton plan, or some labor union leaders and local government interests that were strongly in favor of the plan, are included in the lobbying phase along with the decision makers. Then, when the lobbying game ends (according to the model’s rules), the analysis moves to the next phase. Because of the pulls and tugs during the lobbying period, many players’ positions will have shifted. They will have responded to offers of compromise or to coercion or to the anticipation of such pressures. So at the end of that first game, the decision makers move on to the next phase, but not with their original positions on individual health care issues intact. They move on at whatever position the model predicts they will hold when the lobbying game ends.
The next phase then pits just the decision makers against each other. Gone are labor union leaders, the AMA, the media, the Blues, and local and state governments, and gone is Hillary Rodham Clinton. Sure, she had influence in the lobbying phase, but she didn’t get to vote in Congress. From the model’s perspective, whatever whispers there might have been between her and President Clinton ended with the lobbying phase. He, and others, had ample opportunity in the first phase to succumb to, adjust to, or resist her arguments.
The second phase predicted the passage of a comprehensive bill in both the House and the Senate. It also predicted that the bill that would come before President Clinton was one he could easily have signed, although it would have been much altered from the legislation sought by Hillary Clinton. So there was little need in this case to do a further analysis to work out the negotiations between the House and Senate leadership over the exact contents of the proposed legislation, and there was no need to do a detailed study of the risks of veto and the prospects of overriding a veto. It just wasn’t an issue.
The numbers having been crunched, four results popped out of the analysis as being crucial to understanding where health care reform was headed. First, Hillary Clinton was an unusual stakeholder, not because she was First Lady, but because she showed the characteristics of someone who was content to fail while sticking to her principles. Despite pressure on her from every side, she was nearly immovable on each and every one of the issues I examined. This is a characteristic that is rarely seen in democratic politics (although many perceived this as the bargaining approach, really a bullying approach, used by George W. Bush). Sure, I had seen a rigid adherence to positions before in other studies I had done. The late Nigerian general Sani Abacha (and I do not mean to compare Hillary Clinton or George W. Bush to him on any substantive grounds—just bargaining style back then) was an important focus of many studies I did. He hardly ever shifted position, but then he didn’t have to. He got to dictate outcomes. Hillary Clinton barely shifted positions either, but, from a practical standpoint, she needed to. All the evidence suggests that her time in the U.S. Senate after her husband’s presidency made her a shrewd judge of when to flex some muscle and when to muster some flexibility. That should serve her well in the world now, but back then the model said she only knew about flexing muscle.
In the language of the times, Hillary Clinton had a tin ear when it came to politics. Fair enough. She had never run for office and had not been a politician. But her rigid willingness to go down in a seeming blaze of glory, but inevitably down nevertheless, hurt the chances of forging compromises with those who saw themselves excluded from the debate. This includes such important interest groups as the American Medical Association, much of the pharmaceutical industry, and others from whom even grudging support would have made selling health care reform much easier. Indeed, the analysis suggested that given the right response from the Clinton health care task force, the AMA was more flexible on many health care issues than was commonly perceived at the time. They could have been brought around to support a program that could have passed the House and Senate.
The second striking result was Bill Clinton’s bargaining style within the model’s logic. There are two ways to maneuver into a winning position. One is to persuade others to adopt your point of view. The other is to adopt theirs. Bill Clinton—in the model’s logic; I don’t know what he was actually doing behind closed doors—was the latter type. This was probably due to the modest degree of salience coupled with a fairly centrist, even slightly right position assigned to him on most health care issues by the expert panel I used to create the inputs for the model.
As the model saw things, President Clinton would sniff out where the strongest coalition was, and he would move close to it. He was like a person with a wet finger stuck in the wind to see which way the breeze is blowing. If Hillary Clinton’s principle at the time can be described as “Back what you believe in, come hell or high water,” Bill Clinton’s principle was “Win, no matter what constitutes winning.” With the benefit of hindsight, writing this a decade and a half later, I think it fits pretty well with what many have come to think of as Bill Clinton’s governing style.
The third striking result was how ineffective many members of Hillary Clinton’s task force were at looking past their personal beliefs. They were more open to compromise than Hillary Clinton, but they were reluctant to take her on, and so they were willing to accommodate the opposition too little to build a bridgehead to victory. In the model’s vocabulary, they gave in to her while failing to realize that they had more potential to change her mind than they thought.
And the fourth result, the really striking result, was that Dan Rostenkowski—that is, the person controlling the purse strings in the purse-string-shaping House Ways and Means Committee—had none of these limitations. He maneuvered skillfully—again, I am talking in terms of the model’s predictions; I don’t know what really went on, I only know how it turned out. He knew how to alter the thinking of other players from Congress. He knew how to reshape the president’s thinking and the perspective of many on the task force. He also handled the opposition lobbyists and interest groups skillfully.
What did the model see in Rostenkowski that was absent from the Clintons or other players? In order for health care reform to matter, it had to be funded, of course, and that was the province in which Dan Rostenkowski exerted the greatest influence. The expert data, not surprisingly, rated him as enormously powerful on questions related to how to pay for health care reform. And he was moderately conservative on this question, wanting to shift most of the cost away from the federal budget. Bill Clinton was perceived to argue for an even more conservative position when it came to paying for health care. So Rostenkowski was seen as the more moderate of the two, and he was believed to be as powerful as the president on this question.
Rostenkowski was positioned at a point that had a great mountain of powerful support behind it, he had enough salience to influence people (but not so much as to come across as excessively intense and committed), and he was surrounded by small, fragmented clusters of influence scattered across many positions with relatively little clout to withstand his pressure. In that environment (according to the model’s logic), Rostenkowski was positioned as a leader who could and would move people to his position. He found the right arguments and the right opportunities and the right targets to cajole or coerce so that the prospective winning position was located close to where he wanted it to be. He didn’t go to the winning position, he brought it to him. Thus, on one health care issue after another, because he exerted so much control over the money, what Rostenkowski wanted, Rostenkowski could pretty much get. Except, ah, except for those seventeen felony counts. They were not part of my analysis, they were truly unforeseen, and they made all of the difference. The felony counts were exogenous shocks—that is, the product of outside, unexamined forces unrelated to health care issues.
The political world and the business wor
ld are vulnerable to unanticipated shocks. With the Rostenkowski experience in hand, I realized I needed to have a way to anticipate unpredictable events so that I could take them into account. But how can you predict the unpredictable? Well, although it is impossible to anticipate unpredictable developments, it is possible to predict how big an “earthquake” is needed to disrupt a prediction. I worked out how to predict the magnitude of disruptions even if I could not know their exact source. We’ll look at how I have since incorporated this element into my work.
How I got to the ever-evolving solution is an interesting story in itself. Around the time that Dan Rostenkowski’s troubles led me to think about random shocks, John Lewis Gaddis, a world-renowned historian, now at Yale University but then at the University of Ohio in Athens, Ohio, invited me to spend a week with him and his students. Gaddis had written a paper in 1992 claiming that international relations theory was a failure because it didn’t predict the 1991 Gulf War, the demise of the Soviet Union, or the end of the cold war. Two well-known political scientists, Bruce Russett at Yale University and James Ray at Vanderbilt University, responded that Gaddis had not taken my predictive rational-choice work into account.1 They contended that this body of work warranted being viewed as a rigorous scientific theory rather than just some exercise in fitting data to outcomes after they were known.
Gaddis paid attention to the claim by Russett and Ray that he had overlooked a relevant body of research. That is what led him to invite me to spend time with him and his students. He was a doubter, and he made no bones about that. Southern gentleman that he is, he couched his doubts in the most civil way; still, I was going to Athens, Ohio, with the expectation on John’s part that his students and he were going to expose my modeling as some sort of hocus-pocus.
I agreed to apply my method to any policy problem that Gaddis and his students agreed on, although I imposed two restrictions. First, they had to know enough about the issue they chose to be able to provide me with the data needed for the model, since I was unlikely to be an expert on the issue and anyway it would be best if the data came from doubters. Second, it had to be an issue for which the outcome would be known over a period ranging from a few months to a year or two, rather than something that would not be known for so long that they couldn’t judge in a timely fashion whether my model’s logic had gotten it right or not. The hope was to look at something we could then correspond about. They would know I had made predictions before the fact, and, the timing being right, they would be able to look back at what I had said and compare it to what actually happened later.
The Predictioneer’s Game: Using the Logic of Brazen Self-Interest to See and Shape the Future Page 17