unSpun

by Brooks Jackson

  And so these men of Indostan, disputed loud and long, each in his own opinion, exceeding stiff and strong, Though each was partly in the right, and all were in the wrong!

  So, oft in theologic wars, the disputants, I ween, tread on in utter ignorance of what each other mean, and prate about the elephant not one of them has seen!

  Unless we want to “tread on in utter ignorance,” like the blind men debating about the elephant, we need to bear in mind that our personal experience seldom gives us a full picture. This is especially true when our experience is indirect, filtered by others.

  Consider what happened during the Gulf War of 1991. During the forty-three-day air campaign, television viewers at home watched “smart bombs” homing in unerringly on their targets time after time. A nation that had lost more than 58,000 members of its armed forces in Vietnam a generation earlier now seemed able to fight a new kind of war, from the air, putting hardly any soldiers at risk.

  Not surprisingly, military briefers were showing the public only their apparent successes, but no amount of skeptical questioning by reporters could undo the enormous impact of what viewers were seeing on their TV screens. Only long after the war did we learn that a lot of “smart” weapons missed their targets and that just 8 percent of the munitions dropped (measured by tonnage) were guided. Contrary to the picture presented on television, nine of ten bombs dropped were old-fashioned “dumb” ones. This was documented in a 1996 report by the General Accounting Office (which has since been renamed the Government Accountability Office). It wasn’t until a decade after the 1991 war that weapons precision actually improved enough to match the false impression created by the selective use of video during Desert Storm. The lesson here is that sometimes what you don’t know or haven’t been told is more important than what you have seen with your own eyes. We humans have a natural tendency to overgeneralize from vivid examples.

  The reason we should trust the GAO’s report over the evidence offered by our own eyes is that the GAO had access to all the relevant data, including the Pentagon’s bomb-damage assessment reports, and it systematically weighed and studied that mass of information. Also, the GAO is an arm of Congress with a reputation for even-handed evaluation and for casting a skeptical eye on the claims of agencies (such as the Pentagon) that seek taxpayer money for their programs. The GAO study relied on evidence; what Pentagon briefers showed Americans during the war was a collection of anecdotes—and carefully selected anecdotes at that. The public saw few if any “smart” bombs miss, when in fact they missed much of the time. War was made to seem less messy, less morally objectionable, than it really is.

  The Great Fertilizer Scare

  Without real data and hard evidence, it’s easy to be led astray in all sorts of matters, especially when a dramatic story involving well-known figures captures our attention. The Great Fertilizer Scare of 1987 will illustrate. It began when a former San Francisco 49ers quarterback, Bob Waters, was diagnosed with amyotrophic lateral sclerosis, also known as Lou Gehrig’s disease, and it was reported that two other players from the 1964 team also had ALS. Soon newspapers were carrying a story saying a common lawn fertilizer, Milorganite, which a groundskeeper recalled using on the 49ers practice field, was suspected as a cause.

  Milorganite is sewage sludge, recycled by the Milwaukee Metropolitan Sewerage District. It is dried for forty minutes at between 840 and 1,200 degrees Fahrenheit, a process that kills bacteria and viruses, but it does contain tiny amounts of certain elements, including cadmium, which some researchers at the time suspected might be a cause of the disease. A Milwaukee newspaper dug up the fact that two former Milorganite plant workers also had died of ALS (out of 155 total deaths among workers, from all causes) and that 25 ALS patients in the Milwaukee area claimed to have been in contact with Milorganite. Time magazine and The Associated Press, among others, also ran stories citing a “possible” link between Milorganite and ALS.

  But tragic as slow death from ALS certainly is, the story of the three celebrity football players is a bit like our tale of the great crow fallacy. An EPA epidemiologist who headed a team to study the matter, Patricia A. Murphy, said the ALS-Milorganite connection “has been blown up out of some groundskeeper’s imagination.” She found no evidence of an increase in ALS in the Milwaukee area or Wisconsin as a whole, and concluded that “the anecdotal stories linking ALS and Milorganite are purely that, i.e., anecdotal with no basis in scientific fact.” Dr. Henry Anderson, Wisconsin’s state environmental epidemiologist, agreed. He reported that scientific evidence against Milorganite was lacking and recommended that the state not give high priority to further study. The three 49ers might not even have been exposed to Milorganite. No records were ever found to support the groundskeeper’s recollection. Even if the three players had been exposed to Milorganite, they appear to constitute what statisticians call a random cluster, meaning an unusually large grouping that occurs purely by chance. Epidemiologists are trained to look at the whole picture without being misled by such coincidental clusters. Sooner or later, if you toss a coin enough times, it will come up heads ten times in a row. The odds of it coming up heads on any given toss, however, are always 50–50. And the odds that Milorganite had anything to do with ALS are somewhere between very low and zero.

  Mitch Snyder’s “Meaningless” Numbers

  Real evidence, unlike attention-grabbing anecdotes, is generated by systematic study. But not all studies are created equal, and some hardly deserve to be called studies at all. Consider a claim made in 1982 that more than 3 million Americans could be homeless. It was widely reported as an estimate that 3 million were homeless, and it was widely believed. Critics of the Reagan administration saw the figure as evidence that the Republican president’s policies were creating a social calamity on a scale not seen since the days of the Dust Bowl and the Great Depression. Had 3 million been an accurate number, it would have meant that one American in every seventy-seven was living on the street.

  In fact, the figure was close to an outright fabrication. The source was Mitch Snyder, a former advertising man and ex-convict (he served a federal prison term for grand theft, auto) who had turned himself into an advocate for the homeless and an unrelenting detractor of Ronald Reagan. His methods were hardly scientific. In 1980, he and others at the Community for Creative Non-Violence had called up one hundred local clergy, city officials, and others involved in aiding the homeless in twenty-five cities, asking them for a quick estimate of the number of homeless persons in their locality. But an estimate is not a count, and some estimates are more reliable than others. For one thing, CCNV rejected estimates that Snyder deemed too low. On this highly subjective basis, the group concluded that one percent of the entire U.S. population, or about 2.2 million people, were homeless at the time of the survey. Then, after a recession hit the economy, CCNV said, “We are convinced the number of homeless people in the United States could reach 3 million or more in 1983.” But that was nothing more than their guess piled on top of a dubious “average” of their cherry-picked estimates.

  While testifying before the House Subcommittee on Housing on May 24, 1984, Snyder was challenged to support his estimate; he all but admitted that he had pulled the number from thin air. He said: “These numbers are in fact meaningless. We have tried to satisfy your gnawing curiosity for a number because we are Americans with western little minds that have to quantify everything in sight, whether we can or not.” But few reporters took note; instead, many repeated the “meaningless” 3 million estimate for years without conveying any sense of its spurious basis. More than seven years after Snyder confessed that his number was “meaningless,” for example, the CBS reporter John Roberts stated flatly that there were “more than three million homeless in America.”

  LESSON: Not All “Studies” Are Equal

  SOME QUESTIONS TO ASK WHEN THINKING ABOUT A DRAMATIC FACTUAL claim:

  • Who stands behind the information?

  • Does the source have an ax to grind?


  • What method did the source use to obtain the information?

  • How old are the data?

  • What assumptions did those collecting the information make?

  • How much guesswork was involved?

  Today, we can say with some confidence that homeless people in the United States number in the hundreds of thousands, not in the millions. This is still a lot of people without a home, but it’s a fraction of what Snyder claimed.

  In late March 2000, the U.S. Census Bureau counted 170,706 persons in homeless shelters and soup lines and at a number of open-air locations, such as under the Brooklyn Bridge, where homeless persons were known to gather. The Census Bureau says that figure shouldn’t be taken as a count of all the homeless—it concedes, for example, that it missed anyone who was not using a shelter on that early spring day, or who was sleeping in an open-air location other than those checked. But at least we can be certain that those 170,706 homeless persons were actually counted.

  How many were missed is still open to question. Martha Burt, an expert at the Urban Institute who has studied homelessness for years, estimates that homeless people number no fewer than 444,000 and that the figure is probably closer to 842,000. Her estimates are projections based on an unprecedented, onetime Census Bureau survey of thousands of programs providing services to the homeless in seventy-six cities, suburbs, and rural areas. In October and November 1996, the bureau counted the persons being served at a random sample of those service locations and interviewed a random sampling of 4,207 clients to get additional information. The survey wasn’t intended or designed to produce a national estimate of the homeless population, and the Census Bureau didn’t attempt to derive one. But Burt made a few assumptions and calculated that nationally the number would have been 444,000 homeless adults and children using services on an average week in October and November, the months in which the Census Bureau conducted its head count. Burt also estimated that 842,000 homeless adults and children would have used services nationally on an average week in February, when the weather is much colder. Her estimates are just that—estimates—but they extrapolate from solid data, on the basis of assumptions that are fully disclosed.

  Something else gives us confidence in Burt’s estimate that between 444,000 and 842,000 Americans are homeless: several other studies using different but still systematic methods have come up with figures that are generally in the same range. When different methods arrive at similar estimates, those estimates are more credible. We call this “convergent evidence.”

  A Taxing Argument

  As a general rule, the source of evidence matters. Snyder’s figures should have been viewed with a critical eye from the start both because he was lobbying for more federal spending to aid homeless people and because he was a bitter critic of the Reagan administration. While there are certainly scrupulously honest advocates out there, it’s clear that Snyder’s figures were “data in the service of ideology.” But we can place more trust in studies and data from sources that have no horse in the race, and who haven’t deceived us in the past. Such sources often give us a picture very different from the one painted by self-interested people who are trying to sell us something, whether it’s a product or a policy.*2 Examine, for example, the dubious charges made against the federal estate tax by the lobbyists and partisans who sought its repeal.

  * * *

  “Death Tax” Bunk

  RADIO AD:

  When you die, the IRS can bury your family in crippling tax bills. It can cost them everything. What’s worse, the death tax is a double tax on all you’ve worked to build.

  (American Family Business Institute, June 2005)

  PRESIDENT GEORGE W. BUSH:

  We also put the death tax on the road to extinction because farmers and small business owners should not be taxed twice after a lifetime of work.

  (Radio address, January 21, 2006)

  * * *

  Farmers and small businesses were under attack in the year 2005, if a radio ad from that year is to be believed: “When you die, the IRS can bury your family in crippling tax bills. It can cost them everything.” That claim was in a radio ad that ran in eight states in 2005, part of what the sponsor said was a $15 million campaign to repeal the tax permanently. The ad was paid for by the American Family Business Institute, a group made up of about 500 businesses and including three billionaires, according to organizers. Others known to have funded anti-estate-tax lobbying include the billionaire Mars candy and Gallo wine clans.

  The same claim had been picked up by the Republican party and had become part of its antitax orthodoxy. President Bush repeated it throughout his tenure of office, saying six weeks after his inauguration that Congress should “eliminate the death tax so family farmers aren’t forced to sell their farms before they want to.” But the notion that farmers are forced to sell out and that families can lose “everything” is simply false.

  The fact is, as the nonpartisan Congressional Budget Office said in a report issued in July 2005: “The vast majority of estates, including those of farmers and small-business owners, had enough liquid assets to pay the estate taxes they owed.” In other words, the “vast majority” had no need to sell any assets at all—let alone “everything”—to pay estate taxes. The CBO said that returns filed in 2000 showed only 138 farmers and 164 family business owners left estates without enough liquid assets to pay their estate taxes. And the CBO estimated that those numbers would fall to fifteen farmers and sixty-two family business owners by the year 2006, when only estates valued at $2 million or more would be subject to any tax.

  The CBO was putting matters cautiously. The New York Times reported in 2001 that one of the leading advocates for estate tax repeal, the American Farm Bureau, “said it could not cite a single example of a farm lost because of estate taxes.” The Farm Bureau, a huge lobbying organization that calls itself “the voice of agriculture,” then went scrambling to find such an example. Within days of the Times report, the organization’s president issued an internal bulletin saying: “It is crucial for us to be able to provide Congress with examples of farmers and ranchers who have lost farms or have had to sell off portions of their land that makes [sic] the remaining parcel ‘inefficient,’ due to death taxes.” The memo was quoted by Congressional Quarterly Daily Monitor, which also said no examples had been found. When we called the Farm Bureau in early 2006, they weren’t very happy to hear from us, and they still could not point to a farm lost to pay estate taxes.

  That anyone could “lose everything” to the estate tax is a logical impossibility, anyway. As of 2006, there is no tax at all on the first $2 million of any estate (effectively, $4 million for a couple that takes certain estate planning steps). The maximum tax rate is 46 percent (and set to go down to 45 percent in 2007) on anything above those amounts. Far from losing everything, heirs keep most.

  We’re not arguing for or against estate taxes. We are saying that hard data from tax returns, analyzed by a nonpartisan body noted for its consistent use of reliable methods, show that one of the principal claims made against the estate tax is false. There’s another argument, with which economists generally agree, that taxing wealth reduces the incentive to invest and in turn holds back economic growth and job creation. That argument isn’t very popular with the public, but at least it’s honest. We can’t say that about the radio ad warning that “your family” could be “buried” by the tax. In fact, the estate tax fell upon only 1.17 percent of the estates of adults who died in 2002, according to the IRS Statistics of Income. In other words, nearly 99 percent of the people who heard those radio ads wouldn’t be touched by the tax at all. With respect to them, the ads were flatly untrue.

  LESSON: Saying It Doesn’t Make It So

  CRITICS OF THE ESTATE TAX BASED THEIR YEARS-LONG CAMPAIGN on the idea that repealing the tax would primarily benefit farmers and small-business owners. But most of the wealthy few who pay the tax aren’t farmers or small-business owners by any definitio
n. And the vast majority of small-business owners and farmers will never pay a penny of estate tax. Constant repetition of the claim may have caused people to believe it, but repetition didn’t make it true.

  An Abortion Distortion

  Bogus studies pop up in politics all the time. In 2005, Democrats were gloating over a macabre statistic that would have embarrassed President Bush, if it had been true. They said the number of abortions had gone up since this anti-abortion president took office, and they blamed his economic policies for driving poor women to end pregnancies rather than bear children whom they would be unable to support. Citing “the draconian policies” of Republicans, John Kerry proclaimed, “And do you know that in fact abortion has gone up in these last few years?” The Democratic National Committee chairman, Howard Dean, even went so far as to quantify the purported increase during an appearance on NBC News’s Meet the Press: “You know that abortions have gone up 25 percent since George Bush was president?” In fact, the respected Guttmacher Institute, whose figures on abortion trends are systematically gathered using a disclosed method and are used by both sides—reported that the number of abortions performed in the United States had continued its twenty-year decline after Bush took office. According to the institute’s most recent figures, published in 2006, abortions declined in each of the first three years of Bush’s tenure, for a total drop of nearly 2 percent.

  The false claim that abortions were rising originated with a flawed “study” by Glen Harold Stassen, who is neither a statistician nor a healthcare expert, but an ethics professor at Fuller Theological Seminary. Stassen’s article originally appeared not in any scientific journal but in a liberal Christian publication, Sojourners, in October 2004. “Under President Bush, the decade-long trend of declining abortion rates appears to have reversed,” he wrote. “Given the trends of the 1990s, 52,000 more abortions occurred in the United States in 2002 than would have been expected before this change of direction.” Stassen cited data from sixteen states. The claim was picked up and repeated uncritically on many Internet web logs, both liberal and conservative, and by Democrats including Kerry and Dean.