Our additive total reached 1,000 by New Year. Weekly recoveries then went up from 300 in February to 700 in April and the total passed 10,000 in mid-May. The weekly rate stayed over 700 (with one peak over 1,000) until end-July. When we then began to close down, we had recovered over 20,000 additives. Over 4,000 messages, half of the available traffic, had been deciphered and passed to the bookbuilders and translators. Our recovery rate, especially in the later stages, compared not unfavourably with Washington’s on tasks of similar size. But the time and effort it took for us to recover those 20,000 additives adds an awesome perspective to the immensity of what Washington had been achieving in the face of all 1944’s problems.
The enhancement of our staffing from the time of the Agreement – modest as it was – and our greatly increased access to Hollerith (the Freeborn test) demonstrate how our work mattered at the top at this time. They wanted us to succeed on the assigned tasks. To what extent did this reflect the immediate need to defeat Japan, and to what extent the bolstering of Bletchley Park’s case for keeping a place at the top table in the post-war intelligence world? The question seems likely to remain unanswered.
* Henry Reed, Collected Poems, O.U.P., Oxford, 1991
10
MOST HELPFUL AND CO-OPERATIVE: GC&CS AND THE DEVELOPMENT OF AMERICAN DIPLOMATIC CRYPTANALYSIS, 1941–2
DAVID ALVAREZ
Introduction
In Chapter 10, David Alvarez describes how GC&CS helped the US Army’s Signal Intelligence Service (SIS), as it then was, to develop its diplomatic codebreaking capability significantly during the war. Under an inter-service agreement, the US Army had sole responsibility for diplomatic codebreaking, although this was later modified to enable the US Navy to share the load. The SIS was in favour of Sigint co-operation with Britain as early as August 1940, whereas the US Navy codebreakers, headed by Commander Laurance Safford, had then set their face against any collaboration on codebreaking.
In February 1942, following a review of GC&CS’s functions, Alastair Denniston was moved sideways to become the Deputy Director (Civil), in charge of GC&CS’s diplomatic and commercial sections, which moved to Berkeley Street and the nearby Aldford House, respectively, in London. Edward Travis replaced him as the operational head of Bletchley Park, with the title of Deputy Director (Services).
Despite having relatively few staff, GC&CS’s diplomatic section produced huge numbers of decrypts. In 1940, a staff of about sixty-five read 70,000 signals out of the 100,000 received, although only 8,500 were actually circulated. The Foreign Office received all of them, with other important clients being the service Ministries and MI5, where the recipient was Major Anthony Blunt, one of the notorious ‘Cambridge Five’ group of Soviet agents. In 1940, MI5 was sent only 1,200 of the ‘BJs’, as the diplomatic decrypts were known, on account of their distinctive blue jackets, but by 1943 it was the diplomatic section’s second biggest client, receiving 9,300 BJs compared with the 13,000 sent to the Foreign Office. It would be of considerable interest to know how many Blunt passed on to the Russians, and how they used them. The Russians have claimed that they too reconstructed the Purple machine in late 1941, although they adduced no evidence. If, as is quite probable. Blunt received the Purple decrypts solved by the British, they would have been valuable to the Russians as cribs in solving Purple. But even if they were not used for cribs, the value of one Purple decrypt to the Russians would have been priceless. On 27 July 1942, when battles were raging on Russia’s western front, the Japanese Foreign Minister, Togo Shigenori, informed Japan’s ambassador in Berlin, Oshima Hiroshi, that Japan would not attack the Soviet Union. Did Blunt receive the decrypt of that message? We shall have to await the release of the BJs for 1942 to learn the answer.
In contrast to the major difficulties that emerged in 1943 over the US Army’s desire to attack Heer and Luftwaffe Enigma, co-operation between Britain and the United States on diplomatic codebreaking was remarkably trouble-free from its start in early 1941. Partly for that reason, and partly no doubt because of the range of countries potentially involved, no formal agreement about diplomatic Sigint was ever concluded between the US War Department and GC&CS. Inevitably there were misunderstandings from time to time, but they were resolved, in no small measure due to the wise approach adopted by Alastair Denniston, who was wholeheartedly in favour of Sigint co-operation with the United States. Denniston was a man of vision on this issue, just as he had been in 1938 when he recruited Alan Turing and Gordon Welchman to join GC&CS when war with Germany was declared. Sadly, there has been insufficient recognition of his vital role in laying the foundations of GC&CS’s wartime successes and in paving the way for Britain’s important Sigint alliances with the United States.
RE
Signals intelligence has fully emerged from the historical shadows of the Second World War. With the release in the past decade of almost 1.5 million pages of cryptologic materials into American and British archives, historians have acquired a documentary base upon which to construct an important story, most of whose chapters had remained among the still-guarded secrets of the war. The new materials have enhanced our understanding of well-known episodes in the signals intelligence history of the war, such as the Battle of Midway and the Battle of the Atlantic. More importantly, they have also revealed previously unknown or only dimly perceived episodes of that history, such as the highly successful Anglo-American cryptanalytic effort against the wartime diplomatic codes and ciphers of a range of friendly, neutral and hostile powers. The revelations have especially surprised American historians who, though well aware of their country’s wartime success against Japanese cryptosystems, have been astounded to learn that by the end of the war the US Army’s Signal Security Agency, the organization responsible for diplomatic cryptanalysis, was reading the secret communications of more than sixty governments and organizations. They have been even more surprised to discover that this impressive record, which catapulted the United States into the front rank of intelligence powders, was achieved largely because of significant advice and assistance from Britain’s signal intelligence service, the Government Code and Cypher School (GC&CS).
In the decade following its creation in 1930, the US Army’s codebreaking organization, then known as the Signal Intelligence Service (SIS), had a very limited horizon. Until 1938, the SIS studied the codes and ciphers of only one country: Japan. That country was selected, not because it monopolized American diplomatic attention in the 1930s (it didn’t), but because Tokyo’s communications represented the path of least resistance for a small and inexperienced cryptanalytic organization. With few personnel (the original staff consisted of five men and one woman, none of whom had any training in cryptology), William Friedman, the pragmatic civilian director of the service, decided to concentrate on the cryptosystems of one country. Without any guidance from the State or War departments, he selected Japan because the SIS had inherited an archive of Japanese cryptologic material from a predecessor agency, the so-called Cipher Bureau that had solved several Japanese ciphers during its brief existence in the 1920s.
Friedman trained his novice cryptanalysts with practical problems from the files of intercepts and solved and unsolved ciphers in the Japanese collection of the defunct Cipher Bureau. Once satisfied that his pupils possessed some basic skills, he tested them against the small stream of current Japanese messages that trickled into the War Department from the army’s primitive network of radio intercept stations. Apt pupils, a good teacher, and relatively simple ciphers combined to produce a formula for success. By 1935 the SIS had solved five Japanese diplomatic ciphers; by 1938 it had solved another four, including the so-called Red cipher machine used by the Japanese foreign ministry for its most secret messages.
The Signal Intelligence Service did not abandon its exclusive focus on Japan until the spring of 1938 when, shortly after the Anschluss and just as Berlin was opening a political offensive against Czechoslovakia that would culminate in September in the Munich agreement, Willia
m Friedman directed Solomon Kullback, one of his original recruits, to open a study of German ciphers. Later that year, Italy and Mexico were added to the target list under the direction, respectively, of Abraham Sinkov, another of Friedman’s original recruits, and Herrick ‘Frank’ Bearce, who had joined the service in 1936.
Unfortunately the triumph against Japan did not translate into widespread success against the new targets. By early 1939 Frank Bearce had solved at least one Mexican cipher and was making rapid progress against three others. German and Italian systems, however, proved harder nuts to crack. The German foreign ministry used several cryptosystems and Solomon Kullback’s small team had decided to focus on an unenciphered code, the Deutsches Satzbuch (DESAB). When Germany sparked the Second World War by attacking Poland on 1 September 1939, this book had been sufficiently reconstructed that messages in the code could be read. Unfortunately, DESAB carried only 5 per cent of Berlin’s diplomatic traffic and then only low-grade consular and administrative messages. The high-grade ciphers that protected Berlin’s more important diplomatic communications remained a mystery to the SIS. In the Italian section things were even worse. At the outbreak of war, Abraham Sinkov’s team was studying two systems: an unenciphered code known to the Americans as ‘X’ and an enciphered code known as ‘Trujillo’ (or TR) because most of the intercepted messages in this system were collected on the Rome-Ciudad Trujillo (Dominican Republic) circuit. Neither of these systems was readable.
In autumn 1939 the sound of distant cannons echoed, if only faintly, down the normally somnolent corridors of American military intelligence. Within weeks of the outbreak of war in Europe, Army Chief of Staff General George Marshall approved a proposal to expand the Signal Intelligence Service by hiring twenty-six additional personnel, which would more than double the staff of a service that after nine years still numbered fewer than twenty analysts, translators, clerks and machine operators. While welcome, the additional resources had little impact on operations. There was no attempt to expand the target list beyond the four countries studied before the war. Most of the new staff were assigned to the lagging German and Italian problems, but since they required months of training to reach even a modest level of cryptanalytic skill, their presence did little in the short term to accelerate the effort. Fully a year into the European war, the American signals intelligence programme had advanced only marginally beyond its position in the summer of 1939, and in one area it had actually regressed.
Kullback’s German desk still struggled with its target. The desk continued to read the low-grade messages in DESAB and had discovered that occasionally this basic code was enciphered with reciprocal bigram tables. Unfortunately, this enciphered version (known as Spalierverfahren) was observed only on relatively quiet Caribbean circuits (e.g. Berlin–Havana) and the low traffic volume undermined solution efforts. Kullback’s team had also isolated what it believed to be two different high-grade systems, but had made no progress beyond suspecting that one was enciphered by a string of random additives. The situation in the Italian section was hardly better. By the summer of 1940, Sinkov’s team had reconstructed enough of the ‘X’ code to read most messages in the system, but Rome used the system only for routine administrative traffic, and the intelligence content of the messages was negligible. The team had also identified the encipherment and reconstructed a portion of the codebook for TR, the medium-grade system used by Rome to communicate with its minor Caribbean missions. Traffic in TR was light and only a few messages were read. At least two high-grade Italian systems had been identified, but Sinkov’s team was still working on the encipherment and no messages in these systems were readable.
While progress in the German and Italian sections was disappointing, the situation in the Japanese section was disastrous. Long accustomed to reading all of Japan’s diplomatic traffic, the section had been alarmed at the end of 1938 by decrypts that suggested that Tokyo was preparing to introduce a new cipher machine. Alarm turned to panic when, on 21 February 1939, SIS intercepted three messages from the Japanese legation in Warsaw that should have been readable but were not. Within a week a trickle of unreadable messages became a flood. The Japanese foreign ministry had introduced a new cipher machine to replace Red. It would take some time for the ministry to distribute the new machine (christened Purple by the Americans) and a few minor posts continued to use the old machine well after the outbreak of war, but by 1940 only a handful of Red messages were intercepted each month as Tokyo entrusted its most secret communications to the unreadable Purple machine. The cryptologic window into Japanese diplomacy had closed.
The inability to solve the Purple machine or advance against high-grade German and Italian ciphers meant that in the desperate first year of the war, as Germany defeated and occupied Poland, Denmark, Norway, France, Belgium, the Netherlands and Luxembourg, and threatened to invade Britain, the Signal Intelligence Service could read the high-grade diplomatic traffic of only one country: Mexico. As President Franklin Roosevelt and his State Department frantically (and futilely) sought to formulate a policy that would contain Nazi aggression, stiffen the resolve of threatened nations, and deter Benito Mussolini from leading Italy into the war alongside Hitler, American signals intelligence provided little support. The foreign ciphers accessible to the SIS revealed little beyond the routine and often trivial work of German, Italian and Mexican embassies and consulates: the Italian embassy in Washington informed Rome that the United States intended to open a consulate in Greenland; the Mexican foreign ministry authorized its embassy in Brussels to withdraw to Paris as German armies approached the Belgian capital; the German embassy in Guatemala informed Berlin that its staff had donated a large sum of money to the Reich Red Cross. With Tokyo’s most important messages wrapped securely in Purple, Japanese decrypts, once the pride of the SIS, were now more likely to reveal negotiations for a textile agreement with Peru than the latest perspective on Germany’s intentions toward Britain. The few political messages transmitted to Tokyo in secondary ciphers revealed little that was not also reported by American diplomats and journalists in Europe.
In their warren of cramped offices in the Munitions Building on the Washington Mall, the army codebreakers cast about for a way to regain the cryptanalytic initiative. Increased effort against Purple might result in a breakthrough, but the personnel in the Japanese section were already logging twelve- and fourteen-hour days. Additional personnel might advance operations, but there was no evidence that the notoriously parsimonious War Department was prepared to be generous. In a gesture that revealed the extent of its desperation, the SIS set aside inter-service rivalry and exchanged observations about Purple with its sister service in the United States Navy, OP-20-G. The Navy’s help was welcome, but breakthroughs still eluded the Americans. Then, on 5 September 1940, a message arrived from London that promised an escape from the cryptanalytic wilderness.
After the fall of France, the Roosevelt administration had cast about for a way to support Britain within the limits set by an American public opposed to armed intervention and an American military establishment ill-equipped to support the security of its own country let alone the security of another. In July 1940 President Roosevelt, in consultation with Secretary of War Henry Stimson and Secretary of the Navy Frank Knox, accepted a British proposal that military representatives of the two governments convene a joint staff conference in London. Within a month, an ‘American Military Observer Mission’ arrived in the British capital ostensibly to study ‘standardization of arms’, but really to discuss co-operation between the armed services of the two countries.
On 31 August 1940, the army representative on the American observer mission, General George Strong, startled his hosts (as well as his American colleagues) by announcing that the US Army was working on Axis codes and ciphers and proposing that London and Washington exchange information on their cryptanalytic operations. The British were particularly surprised by Strong’s proposal, since the Royal Navy had been rather curtly rebuffe
d by the US Navy in October 1939 when it suggested sharing information on Japanese naval communications. Having had one door slammed in their face, the British were now pleasantly surprised to have another door opened even before they had knocked. Within days they accepted General Strong’s invitation. On 5 September Strong sent the War Department the telegram that would electrify the Signal Intelligence Service: ‘Are you prepared to exchange full information on all German, Italian, and Japanese code[s] and cryptographic information therewith? Are you prepared to agree to a continuous exchange of important intercept in connection with the above? Please expedite reply.’
The Signal Intelligence Service was more than prepared to pursue any opportunity to advance its lagging cryptanalytic programme. Indeed, even before Strong’s message reached Washington, William Friedman and Colonel Spencer Akin, the military commander of the service, may have already concluded that co-operation with the British promised a short cut to success. About the time that Strong was proposing cryptanalytic co-operation to his astonished British and American colleagues in London, Friedman and Akin were preparing a memorandum, ‘Proposed Exchange Basis with the British’, that recommended the exchange of ‘any and all material that we have on a basis of complete reciprocity’, and explicitly stated that the SIS was interested in information concerning specific foreign codes and ciphers.
It is easy to understand the SIS’s enthusiasm for establishing links to GC&CS. When Akin and Friedman composed their memorandum, Japanese traffic was still America’s only productive source of diplomatic signals intelligence. Unfortunately, even this source had declined significantly in value after Tokyo’s introduction of the yet unsolved Purple cipher machine. The low-grade German and Italian traffic accessible to American codebreakers produced negligible intelligence and would continue to do so as long as Berlin’s and Rome’s high-grade ciphers remained impenetrable. The potentially useful communications of powers such as China, Russia and Vichy France had not even been studied let alone exploited. A connection to the British might provide short cuts to success against all these targets. Given the secrecy that surrounded GC&CS, American codebreakers knew nothing about operations at Bletchley Park, the wartime home of the British codebreakers, but through Friedman they were vaguely aware of Britain’s cryptanalytic achievements during the First World War and assumed that GC&CS was working hard to repeat those successes in the present war. Perhaps the British were already reading high-grade German and Italian ciphers. An exchange of ‘any and all material’ might well produce just the information necessary for an American entry into Nazi and Fascist communications. With luck an exchange might even help the effort against Purple.
The Bletchley Park Codebreakers Page 19