The Theory That Would Not Die

by Sharon Bertsch McGrayne

  A frequentist who tests a precise hypothesis and obtains a p-value of .04, for example, can consider that significant evidence against the hypothesis. But Bayesians say that even with a .01 p-value (which many frequentists would see as extremely strong evidence against a hypothesis) the odds in its favor are still 1 to 9 or 10—“not earth-shaking,” says Jim Berger, a Bayesian theorist at Duke University. P-values still irritate Bayesians. Steven N. Goodman, a distinguished Bayesian biostatistician at Johns Hopkins Medical School, complained in 1999, “The p-value is almost nothing sensible you can think of. I tell students to give up trying.”53

  Jeffreys was making Laplace’s probability of causes useful for practicing scientists, even as Fisher was doing the same for Laplace’s frequency-based methods. The difference was that Fisher used the word “Bayes” as an insult, while Jeffreys called it the Pythagorean theorem of probability theory. As the first since Laplace to apply formal Bayesian theory to a variety of important scientific problems, Jeffreys became the founder of modern Bayesian statistics.

  Statistically, the lines were drawn. Jeffreys and Fisher, two otherwise cordial Cambridge professors, embarked on a two-year debate in the Royal Society’s Proceedings. Jeffreys may have been shy and uncommunicative, but when he was sure of himself he dug in, placidly but implacably. Fisher remained his usual “volcanic and paranoid” self.54 Both men were magnificent scientists, the world’s leading statisticians, and each used the methods best suited to his respective field. Yet neither could see the other’s point of view. Like gladiators of old, they hurled impassioned articles back and forth, criticizing one another and issuing formal rejoinders, probing rebuttals, and brilliant clarifications—until Royal Society editors threw up their hands in exasperation and ordered the warriors to cease and desist.

  After the grand debate, Jeffreys wrote a monumental book, Theory of Probability, which remained for years the only systematic explanation of how to apply Bayes to scientific problems. Fisher complained publicly that Jeffreys makes “a logical mistake on the first page which invalidates all the 395 formulae in his book.”55 The mistake, of course, was to use Bayes’ theorem. Summarizing Jeffreys’s books, Lindley said, “De Finetti is a master of theory, Fisher a master of practice, but Jeffreys is brilliant at both.”56

  The Fisher–Jeffreys debate ended inconclusively. Practically speaking, however, Jeffreys lost. For the next decade and for a variety of reasons frequentism almost totally eclipsed Bayes and the inverse probability of causes.

  First, Fisher was persuasive in public, while the mild-mannered Jeffreys was not: people joked that Fisher could win an argument even when Jeffreys was right. Another factor was that social scientists and statisticians needed objective methods in order to establish themselves as academically credible in the 1930s. More particularly, physicists developing quantum mechanics were using frequencies in their experimental data to determine the most probable locations of electron clouds in nuclei. Quantum mechanics was new and chic, and Bayes was not.

  In addition, Fisher’s techniques, written in a popular style with minimal mathematics, were easier to apply than those of Jeffreys. A biologist or psychologist could easily use Fisher’s manual to determine whether results were statistically significant. To use Jeffreys’s rather opaque and mathematical approach, a scientist had to choose among five nuanced categories: the evidence against the hypothesis is “not worth more than a bare mention” or it is substantial, strong, very strong, or decisive.57 Characteristically, Jeffreys tucked the five categories into appendix B of his book.

  Finally and most important, Jeffreys was interested in making inferences from scientific evidence, not in using statistics to guide future action. To him, decision making—so important for the rise of mathematical statistics during the Second World War and the Cold War—was irrelevant. Others parted at the same divide: a big reason for the feud between Fisher and Neyman, for example, was decision theory.

  All these factors left Jeffreys almost totally isolated from statistical theorists. His link with Fisher was their interest in applying statistics to science. Jeffreys knew Ramsey and visited him as he was dying in the hospital, but neither realized the other was working on probability theory; in any case, Jeffreys was interested in scientific inference and Ramsey in decision making. Jeffreys and de Finetti worked on similar probability issues during the 1930s, but Jeffreys did not even know the Italian’s name for half a century and would have rejected outright de Finetti’s subjectivity. Most statisticians ignored Jeffreys’s book on probability theory for years; he said that “they were completely satisfied with frequency theories.”58 Jeffreys accepted a medal from the Royal Statistical Society, but attended none of its meetings. Geophysicists did not know about his probability work; a surprised geologist once asked Lindley, “You mean that your Jeffreys is the same Jeffreys as mine?”59

  By 1930 Jeffreys was truly a voice in the wilderness. Most statisticians were using the powerful body of ideas developed by the anti-Bayesian trio. Jeffreys’s great book Theory of Probability was published as part of a series of physics books, not statistics books. It also appeared in the last year of peace, just before the start of the Second World War and a new opportunity for Bayes’ rule.

  part II

  second world war era

  4.

  bayes goes to war

  By 1939 Bayes’ rule was virtually taboo, dead and buried as far as statisticians in the know were concerned. A disturbing question remained, though. How could wartime leaders make the best possible life-and-death decisions swiftly, without waiting for complete information? In deepest secrecy some of the greatest mathematical minds of the century would contribute to rethinking Bayes’ role during the uncertain years ahead.

  The U-boat peril was the only thing that ever really frightened Winston Churchill during the Second World War, he recalled in his history of the conflict. Britain was self-sufficient in little other than coal; it grew enough food to feed only one in three residents. But after the fall of France in 1940, Germany controlled Europe’s factories and farms, and unarmed merchant ships had to deliver to Britain 30 million tons of food and strategic supplies a year from the United States, Canada, Africa, and eventually Russia. During the Battle of the Atlantic, as the fight to supply Britain was called, German U-boats would sink an estimated 2,780 Allied ships, and more than 50,000 Allied merchant seamen would die. For Prime Minister Churchill, feeding and supplying his country was the dominating factor throughout the war.

  Hitler said simply, “U-boats will win the war.”1

  U-boat operations were tightly controlled by German headquarters in occupied France. Each submarine went to sea without orders and received them by radio after it was well out in the Atlantic. As a result, an almost endless cascade of coded radio messages—more than 49,000 are still archived—raced back and forth between the U-boats and France. Although the British desperately needed to know where the U-boats were, the messages were unreadable. They had been encrypted by word-scrambling machines, and no one in Germany or Britain thought their codes could be broken.

  Strangely enough, the Poles were the first to think otherwise. A few intelligence officers in Poland, sandwiched as they were between Germany and Russia, realized a full decade before the start of the Second World War that mathematics could make eavesdropping on their rapacious neighbors quite informative. The First World War had made the need for machines to encode radio messages painfully obvious. When an alphabet-scrambling machine was exhibited at an international trade show in 1923, Germany bought some and began introducing complexities to make their codes more secure. The machines were named Enigma.

  And enigmas they were. The Poles spent three years trying unsuccessfully to crack German messages before realizing that automated cipher machines had transformed cryptography. The science of coding and decoding secret messages had become a game for mathematicians. When the Polish secret service organized a top-secret cryptography class for German-speaking mathematics students, it
s star pupil was an actuarial mathematician named Marián Rejewski. He used inspired guesswork and group theory—the new mathematics of transformation—to make a crucial discovery: how the wheels on an Enigma were wired. By early 1938 the Poles were reading 75% of Germany’s army and air force messages. Shortly before their country was invaded in 1939 they invited French and British agents to a safehouse in the Pyry Forest outside Warsaw, revealed their system, and sent an updated Enigma machine to London.

  To an observer, an Enigma looked rather like a complicated typewriter, with a traditional keyboard of 26 letter keys and a second array of 26 lettered lights. Each time a typist pressed a letter key, an electric current passed through a set of three wheels and advanced one of them a notch. The enciphered letter lit up on the lampboard, and the typist’s assistant read the letter off to a third aide, who radioed the scramble in Morse code. At its destination, the process was reversed. The recipient typed the coded letters into his Enigma keyboard, and the original message lit up on his lampboard. By changing the wiring, wheels, starting places, and other features, an Enigma operator could churn out millions upon millions of permutations.

  Germany standardized its military communications with increasingly complex versions of the machines. Approximately 40,000 military Enigmas were distributed to the German army, air force, navy, paramilitary, and high command as well as to the Spanish and Italian nationalist forces and the Italian navy. When German troops invaded Poland on September 1, 1939, battery-powered Enigmas were the key to their high-speed blitzkrieg as field officers in Enigma-equipped command vehicles coordinated, as never before, a barrage of artillery fire, dive-bombing airplanes, and panzer tanks. Most German naval vessels, particularly battleships, minesweepers, supply ships, weather report boats, and U-boats, had an Enigma.

  Unlike the Poles, the British agency charged with cracking German military codes and ciphers clung to the tradition that decryption was a job for gentlemen with linguistic skills. Instead of hiring mathematicians, the Government Code and Cypher School (GC&CS) employed art historians, scholars of ancient Greek and medieval German, crossword puzzlers, and chess players. Mathematicians were regarded as “strange fellows.”2

  The British government and educational systems treated applied mathematics and statistics as largely irrelevant to practical problems. Well-to-do boys in English boarding schools learned Greek and Latin but not science and engineering, which were associated with low-class trade. Britain had no elite engineering schools like MIT or the École Polytechnique. Two years into the war, when government officials went to Oxford to recruit men proficient in both mathematics and modern languages, they found only an undergraduate mathematics major teaching himself beginning German. The government did not even plan to exempt mathematicians from combat. Knowing that their skills would be needed eventually, mathematicians quietly spread word to their colleagues to register with the government as physicists because they at least were considered vital to the nation’s defense.

  Exacerbating the emergency was the fact that the government regarded statistical data as bothersome details. A few months before war was declared in 1939, the giant retailer Lord Woolton was asked to organize the clothing for Britain’s soldiers. He discovered to his horror that “the War Office had no statistical evidence to assist me. . . . I had the greatest difficulty in arriving at any figures that would show how many suits of uniform and how many boots were involved.”3 The Department of Agriculture ignored a study of the fertilizers needed to increase Britain’s food and timber supplies because it thought the Second World War was going to be a nonscientific war and no more data would be needed. Government functionaries also seemed to think that applying mathematics to real life would be easy. When the Ministry of Supply needed to assess new rockets, it gave an employee one week to “learn statistics.”4

  Probability experts were scarce. For a small elite the 1930s had been the golden age of probability theory, the language of statistics. But the majority of mathematicians thought of probability as arithmetic for social scientists. Cambridge, the center of British mathematics, was a backwater in probability. Germany, a leader in modern mathematics and quantum physics, produced few statisticians. And one of the greatest probability thinkers of the twentieth century, Wolfgang Doeblin, was a 25-year-old French soldier fighting for his life as France fell to the Germans in June 1940. The Gestapo was hunting his father, and Doeblin, surrounded and without hope of escape, killed himself to avoid any chance of betraying his parent. Doeblin’s work would one day be crucially relevant to chaos theory and random mapping transformations.

  Oddly, the Allies’ top three statisticians were sidelined during the war. Harold Jeffreys was ignored, perhaps because he was an earthquake specialist and astronomy professor. British security apparently considered Ronald Fisher, the anti-Bayesian geneticist, to be politically untrustworthy because he had corresponded with a German colleague. Fisher’s offers to help the war effort were ignored, and his application for a visa to the United States was rejected without explanation. A chemist calculating the dangers of poison gas succeeded in arranging a visit to Fisher only by claiming he was collecting a horse nearby. As for Jerzy Neyman, he persisted in carrying out full theoretical studies that could lead to a new theorem even though the military desperately needed quick and dirty advice; one of Neyman’s grants was formally terminated.

  With applied mathematicians and statisticians in short supply, wartime data were often analyzed not by statisticians but by actuaries, biologists, physicists, and pure mathematicians—few of whom knew that, as far as sophisticated statistics was concerned, Bayes’ rule was unscientific. Their ignorance proved fortunate.

  Despite the strange reputation of British mathematicians, the operational head of GC&CS prepared for war by quietly recruiting a few nonlinguists—“men of the Professor type”5—from Oxford and Cambridge universities. Among that handful of men was Alan Mathison Turing, who would father the modern computer, computer science, software, artificial intelligence, the Turing machine, the Turing test—and the modern Bayesian revival.

  Turing had studied pure mathematics at Cambridge and Princeton, but his passion was bridging the gap between abstract logic and the concrete world. More than a genius, Turing had imagination and vision. He had also developed an almost unique set of interests: the abstract mathematics of topology and logic; the applied mathematics of probability; the experimental derivation of fundamental principles; the construction of machines that could think; and codes and ciphers. Turing had already spent hours in the United States discussing cryptography in his high-pitched stammer with a Canadian physicist named Malcolm MacPhail.

  After Turing returned to England in the spring of 1939, his name was quietly added to a short “emergency list” of people with orders to report immediately to the GC&CS in the event war was declared. He worked alone that summer, studying both probability theory and Enigma codes. Occasionally he visited GC&CS to talk with a cryptanalyst, Dillwyn Knox, who had already solved a relatively simple Enigma code used by the Italian navy. By the time Germany invaded Poland, Knox and Turing probably understood more about military Enigmas than anyone else in Britain.

  On September 4, the day after England declared war on Germany, Turing took a train to the GC&CS research center in Bletchley Park, a small town north of London. He was 27 but looked 16. He was handsome, athletic, shy, and nervous and had been openly homosexual at Cambridge. He cared little about appearances; he wore shabby sports coats and had dirty fingernails and a permanent five-o’clock shadow. He would devote the next six years to Enigma and to other coding and decoding projects.

  On his arrival in Bletchley Park, GC&CS analysts divided up the Enigma systems, and Turing worked awhile on army codes. By January the English were reading German air force messages. During the first weeks of the war Turing also designed the “bombe.” This was not a weapon in the traditional sense but a high-speed electromechanical machine for testing every possible wheel arrangement in an Enigma. Tur
ing’s bombe, a radical redesign and upgrade of the device invented by the Poles, would turn Bletchley Park into a code-breaking factory. Turing’s machine tested hunches, 15-letter tidbits suspected of being in the original message. Because it was faster to toss out possibilities than to find one that fit, Turing’s bombe simultaneously tested for wheel combinations that could not produce the hunch.

  Turing refined the bombe’s design with the help of mathematician Gordon Welchman and engineer Harold “Doc” Keen. Their prototype, a metal cabinet roughly 7 by 6 by 2.5 feet, appeared at Bletchley Park in March 1940. Some believe the bombe’s design was Turing’s biggest contribution to breaking Enigma.

  Despite the progress made on breaking German air force and army codes, no one at Bletchley Park wanted to tackle the German naval codes, the key to winning the U-boat war in the Atlantic. Of all the branches of the Axis military, Hitler’s navy operated the most complex Enigma machines and security systems. By war’s end, a naval Enigma machine could be set up an astronomical number of ways. According to a Bletchley Park decoder, “All the coolies in China could experiment for months without reading a single message.”6 At any one time the machine could use 1 of 4 reflector combinations (each of which could be set in 26 different ways); 3 of 8 rotors (giving up to 336 permutations); more than 150 billion plugboard combinations; 17,000 possible clip positions around the rotors; and 17,000 possible starting positions (half a million in four-rotor machines). Many of these settings were changed every two days, sometimes every 8 or 24 hours.

  According to Frank Birch, head of the GC&CS naval intelligence branch, superior officers informed him that the “German codes were unbreakable. I was told it wasn’t worthwhile putting pundits onto them. . . . Defeatism at the beginning of the war, to my mind, played a large part in delaying the breaking of the codes.”7 The naval codes were assigned to one officer and one clerk; not a single cryptanalyst was involved. Birch, however, thought the naval Enigma could be broken because it had to be. The U-boats put Britain’s very existence at stake.