Despite such occasional frustrations, Anglo-American co-operation in diplomatic cryptanalysis was generally smooth. Indeed, fruitful wartime collaboration continued long after the Signal Intelligence Service (renamed first the Signal Security Service and then the Signal Security Agency) evolved into a large organization employing thousands of staff and directing a worldwide intercept network. As American diplomatic cryptanalysis matured, GC&CS continued to provide important support usually in response to direct requests for assistance. In early 1943, for example, Berkeley Street passed to Arlington Hall what British codebreakers knew about the wheel wirings of the Enigma machine used by the Swiss foreign ministry. In August of that year, Berkeley Street provided additional information concerning the Swiss machine as well as full information on the Dutch Hagelin diplomatic cipher machine. In 1944, British contributions included full information (often together with reconstructed codebooks and cipher tables) concerning Bulgarian, Burmese, Greek, Hungarian, Iranian and Iraqi diplomatic systems. Such assistance was particularly welcome as Arlington Hall expanded its operations by opening Balkan and Middle Eastern sections.
Both parties were so satisfied with the state of co-operation that disagreements in other areas of signals intelligence collaboration, such as the famous ‘Enigma Crisis’ of 1943, hardly rippled the waters of diplomatic operations. By that time co-operation against diplomatic targets was taken so much for granted that the SIS felt comfortable asking its operational sections to submit lists of what they needed from GC&CS in the full expectation that those needs would be satisfied. Further, the curious omission of any reference to diplomatic targets in the so-called BRUSA Agreement of 1943 – apportioning responsibilities for Axis military traffic – suggests that the informal exchange arrangements in the diplomatic area were so mutually satisfactory and non-controversial as to require no formal delimitation.
As might be expected, though, the collaboration was never perfect. Occasionally (especially towards the end of the war) one party would decline to share information. In the summer of 1944, for example, the Foreign Office hesitated to disclose to the Americans the contents of the so-called Reserved Series of intercepts, revealing foreign diplomats reporting the comments of British officials on various topics, including American policies. Later that year, the Foreign Office became increasingly skittish about Berkeley Street giving the Americans cryptanalytic information regarding certain countries, such as Egypt, that London considered clients. On one occasion GC&CS felt honour-bound to refuse an American request for a Polish codebook because the Poles had given the British a copy under a promise of security. For their part, the Americans were less inclined as the war wound down to share information about South American systems. Less frequently, one party would launch an operation without informing the other, as in February 1943, when the SIS created a highly secret section to study Russian ciphers. Ironically, the Americans remained unaware that in late 1944 GC&CS established its own secret unit to study Russian internal traffic (civil and military) in Sloane Square, London. Of course, signals intelligence methods and successes have always been among the most precious secrets of any government. What is remarkable is not that London or Washington would hesitate or equivocate before exchanging such secrets, but that such hesitancy and equivocation were so rare.
By the end of the war, the United States had developed a large and productive signals intelligence organization that was reading the diplomatic traffic of almost every government in the world from Afghanistan to Yugoslavia. Much of this success was due to the diligence and skill of American cryptanalysts. No small part, however, was played by the British codebreakers at the Government Code and Cypher School. Throughout the war GC&CS provided timely advice and assistance that significantly advanced the American programme in diplomatic signals intelligence. The value of the British contribution was accurately summarized by the US Signal Security Agency in a postwar review of operations: ‘It is doubtful whether success in solution of certain diplomatic systems could have been achieved in time to be useful had not the British supplied the necessary information … The debt of the SSA to GC&CS in shortening the period between the beginning of study and the production of translations was in the case of the diplomatic traffic of certain governments very great indeed.’ Without doubt, the British had been most helpful and co-operative.
11
BREAKING GERMAN NAVAL ENIGMA ON BOTH SIDES OF THE ATLANTIC
RALPH ERSKINE
Introduction
Chapter 11 describes how Hut 8 and its US Navy counterpart broke naval Enigma. Decrypts from Hut 8 were sent to Hut 4 (Naval Section) for translation and analysis, from where they were sent to the Admiralty’s Naval Intelligence Division in Whitehall. However, Hut 8’s relationship with Hut 4 was initially a somewhat uneasy one. Frank Birch, in Hut 4’s German subsection, desperately wanted to help Alan Turing, who he did not regard as ‘a practical man’, to find a way into naval Enigma in 1940. But Birch did not fully understand the complexities involved in breaking Enigma, and Dilly Knox had not had time to explain the process to him in 1939. At one stage, Birch’s anxieties led to a proposal to put Turing under Birch’s wing, which would have been disastrous. Even Knox found that Turing was ‘very difficult to anchor down. He is very clever but quite irresponsible & throws out a mass of suggestions of all degrees of merit. I have just, but only just, enough authority & ability to keep him & his ideas in some sort of order & discipline.’ Birch would have found it impossible to do so.
Although Birch did not at first fully understand how Enigma could be broken, he did appreciate Hut 8’s desperate need for more bombes, and he fought hard to get them. He was understandably frustrated: bombes were said to be expensive and to require ‘a lot of skilled labour to make and a lot of labour to run’. Even their need for more electric power at Bletchley was raised as a problem. Bletchley had not pressed hard enough for more bombes, and Stewart Menzies, GC&CS’s somewhat nominal director, had not taken the trouble to find out about its needs. Birch saw the issue clearly:
It has been argued that a large number of bombes would cost a lot of money. Well, the issue is a simple one. Tot up the difficulties and balance them against the value to the Nation of being able to read current Enigma.
It took many long and bitter battles, fought mainly by Edward Travis and Birch, before GC&CS acquired more bombes. Later, British manufacturing capacity was so over-extended that, when the Kriegsmarine applied the four-rotor version of Enigma to most of its Enigma ciphers during the course of 1944 (having already done so for its Atlantic U-boat cipher in 1942), Hut 8 had to rely largely on US Navy bombes for its attacks on these ciphers. Fortunately, good cable communications with OP-20-G enabled Hut 8 to use the OP-20-G bombes ‘almost as conveniently as if they had been at one of our outstations 20 or 30 miles away’. And Hut 8’s dependency on OP-20-G’s hardware had indirect benefits in the long run: Hut 8’s virtual partnership with OP-20-G on naval Enigma helped to pave the way for the important postwar UK-USA pact on Anglo-American Sigint co-operation.
RE
Like all navies in the Second World War, the Kriegsmarine had to rely extensively on radio communications. In particular, U-boat pack tactics in the Atlantic were completely dependent on the use of radio. The Admiral Commanding U-boats – Befehlshaber der U-Boote (BdU) – directed the packs on the basis of sightings and other reports from the U-boats, which he collated with all the other intelligence available to him. To protect its signals, the Kriegsmarine employed the three-rotor Enigma machine, M3, with the three additional naval rotors, VI, VII and VIII, although it also used some manual ciphers, when security was not all important. At least twenty naval Enigma ciphers came into service during the war. The principal cipher in 1940 and 1941, Heimisch (later renamed Hydra, and called Dolphin by Bletchley Park), was used by U-boats and surface ships in home waters, including the Atlantic. Some naval ciphers, such as Ausserheimisch (Foreign Waters, later known as Aegir - codenamed Pike by Bletchley) were never broken durin
g the war, although OP-20-G (the US Navy’s codebreaking unit) devoted much effort to attacking Pike in 1944. In addition to Allgemein (general) keys, most naval Enigma ciphers had doubly enciphered Offizier (Officer) keys, which were especially difficult to break. A few had Stab (Staff) keys, which Hut 8 (Bletchley Park’s naval Enigma section) broke on only one occasion.
Despite the fact that the Poles had given GC&CS a reconstructed Enigma and rotors I to V in August 1939, probably only two people at Bletchley Park believed in the autumn of 1939 that naval Enigma could be broken: Frank Birch, the head of the German naval subsection, and Alan Turing. Birch thought it could be broken because it had to be, even though Alastair Denniston, the operational head of Bletchley, had told him at the outbreak of the war that ‘all German codes were unbreakable’. Turing had become interested in naval Enigma when he arrived at Bletchley in September 1939, ‘because … I could have it to myself. Despite Denniston’s forebodings, Turing solved the complex naval indicating system (which is described in Appendix III) in December 1939, using some Polish decrypts for 1 to 8 May 1937. However, since he did not have the bigram tables used with the system, he could make no progress against wartime traffic. He therefore decided to tackle signals that had been intercepted in November 1938. After two weeks’ work, five days’ traffic was broken. It was found that six Stecker (plugboard connections) were still being used, and that no letter was steckered for two days in succession, which eliminated twelve letters on each second day. Turing hoped that this would allow him to make progress quickly, but he had made little headway by May 1940.
New wartime rotors, VI and VII, were recovered from the crew of U-33 in February 1940, but Dolphin proved much more resistant to attack than the Luftwaffe Enigma ciphers. In May and June 1940, using papers captured from the patrol boat Schiff 26, Hut 8 solved six days of April Dolphin traffic, with help from the first British bombe, ‘Victory’. Rotor VIII was captured in August 1940, after which Bletchley held all eight rotors. In November, after months of work, Hugh Foss broke the key for 8 May, which became known as ‘Foss’s day’, using the bombe. The key for 7 May was quickly broken, but only about three other daily keys were solved before February 1941.
None of the ten or so Dolphin daily keys solved in 1940 was broken in time to provide operationally useful intelligence. Hut 8 faced two main problems in trying to break Dolphin after the first bombe with Gordon Welchman’s diagonal board entered service in August 1940: first, it lacked cribs, which were needed on a daily basis to provide menus for the bombes. But none would be available until it had read a substantial amount of traffic, which it could not hope to break. Secondly, GC&CS had only two bombes at the end of 1940, which were seldom available for naval use. Birch complained bitterly that Hut 8 was not getting ‘fair does’, but to little avail. Without a short cut, M3’s eight rotors imposed an impossibly heavy load on the bombes, since up to 336 rotor combinations had to be tested, taking around eighty hours (excluding the time to change rotors) – almost six times longer than a comparable attack on the sixty rotor combinations involved in Heer or Luftwaffe Enigma. The resulting delay of four days or more would have rendered the decrypts useless for operational purposes. Moreover, at least two bombes would have been constantly required, and Bletchley could not spare them from other work for so long. Hut 8 therefore required some method of reducing the number of rotor orders to be checked.
Turing had in fact already invented a Bayesian probability process called Banburismus to solve the difficulty. Banburismus reduced the number of M3 rotor orders to be tested from 336 to a manageable number, usually between forty and sixty, by ascertaining which rotors were in M3’s right-hand and middle slots. However, to use Banburismus Hut 8 needed a complete set of the bigram tables for encoding Dolphin message indicators. Without the tables, Hut 8 had no prospect of aligning Dolphin signals ‘in depth’ by correctly superimposing cipher texts enciphered with the same key, which was the basis of Banburismus. However, Hut 8 lacked both the bigram tables and enough solved traffic to reconstruct them. To solve Dolphin it required an extensive corpus of Dolphin plain traffic. This was a singularly vicious circle from which Hut 8 could escape only if the Royal Navy captured the tables or Enigma key-lists to allow Hut 8 to read the traffic in order to recreate the tables.
On 12 March 1941, Hut 8 received an Enigma key-list for February 1941, which had been snatched from the German armed trawler Krebs during a commando raid in the Lofoten Islands. Hut 8 had read most naval Enigma for February and had broken about eight days’ April traffic by 10 May. Although the decrypts were not available in time to be operationally useful to the Admiralty’s Operational Intelligence Centre (OIC), they enabled Hut 8 to reconstruct the ‘Bach’ bigram tables then in force.
Weather signals were transmitted daily by designated Atlantic U-boats, as an essential part of the German war effort, since so much of European weather moves from west to east. The signals were encoded on the Wetterkurzschlüssel, a short signal book, in order to shorten them as a precaution against shore high-frequency direction-finding (HF-DF), before they were enciphered on Enigma. The weather data were then rebroadcast by the powerful Kriegsmarine transmitter at Norddeich (call sign DAN), after being enciphered in the Kriegsmarine manual cipher called ‘Germet 3’ by Bletchley (and also known by it as ‘the DAN meteorological cipher’). Bletchley Park’s meteorological subsection in Hut 10 broke the DAN meteorological cipher from February 1941 onwards. On 10 May 1941, Hut 8 received the 1940 edition of the Wetterkurzschlüssel, which had been seized from the weather ship München in a specially planned operation. Hut 8 could now reconstruct the exact plain-text of the encoded weather short signals from the U-boats, giving it a further invaluable source for bombe menus.
Enigma key-lists for June and July seized from München and another weather ship, Lauenburg, proved a godsend. The resulting decrypts familiarized Hut 8 with naval Enigma sufficiently for it to solve the August Dolphin keys with an average delay of fifty hours. The lists also greatly eased the task of reconstructing a set of new bigram tables, codenamed ‘Fluss’, which came into force on 15 June. Hundreds of bombe runs would otherwise have been required to help build up the new tables. Since there were then only about six bombes. Hut 8 would probably have been unable to break naval Enigma currently for a further two months – until October – but for the key-lists. The traffic also enabled Hut 8 to assess the extent to which the Kriegsmarine was sending dummy signals consisting of nonsense, in order to defeat traffic analysis: the signals would otherwise have falsified many of its language statistics, which were a vital part of Banburismus.
Despite its new-found knowledge, Hut 8 was unable to solve the Dolphin keys for 1 to 6 August and 18 and 19 September. However, those were the only days on which it failed to break Dolphin during the rest of the war. In finding naval keys, Hut 8 was helped because the inner settings (the rotor mix and ring settings) were changed only every two days, presumably because only officers were permitted to alter the inner settings, saving Bletchley a considerable amount of bombe time. In addition, since Hut 8 knew the rotor order for the second day, it did not have to carry out Banburismus, which was a time-consuming process, on the second day. This almost halved its work but, more importantly, it greatly speeded up the solution of Dolphin for second days. If a crib was available on the second day, a few bombe runs could therefore find the settings very quickly, since only a single rotor order had to be tested. After August 1941, Hut 8 broke most Dolphin traffic within thirty-six hours during the remainder of the war. Hut 8 took about three days to solve the settings for the first of a pair of days’ traffic in August, and under twenty-four hours for the second day. In October 1941, Dolphin took seventy-five hours to solve on the first day of a pair, but only a few hours on the second.
The June and July decrypts also enabled GC&CS’s Naval Section in Hut 4 to spot a breach of cipher security in the inter-connected German signal nets. Because some small Kriegsmarine units were not issued with Enigma, signals were
sometimes enciphered on a manual system known as the Werftschlüssel (dockyard cipher), as well as Enigma. Hut 4 first penetrated the Werftschlüssel in mid-1940 and was breaking it more or less currently by March 1941. The deciphered Werft versions of the signals provided Hut 8 with a second source of cribs. On occasion, minelaying operations (known as ‘Gardening’) were carried out by the Royal Air Force to provide Hut 8 with Werftschlüssel cribs. The Kriegsmarine then sent signals in Dolphin and the Werftschlüssel about the re-opening of the relevant sea lanes after sweeping them for mines.
Shark (codenamed Triton by the Germans), a special cipher for the Atlantic and Mediterranean U-boats, was introduced on M3 as an interim security measure on 5 October 1941, but only thwarted Hut 8 for a few days. When Hut 8 initially could not solve Shark for a few days in early October, it found, by using a re-encodement, that Shark now had a completely different Grundstellung from Dolphin. Previously the U-boats had merely used the reverse of the Dolphin Grund (e.g. DFT instead of TFD). The change made little difference to solving Shark, but it was a harbinger of the threat to come.
On 29 November the bigram tables and Kenngruppenbuch (known to Hut 8 as ‘the K book’) used with them changed. However, the fifteen bombes available in December helped Hut 8 to build up the tables quite quickly. Turing was about to start the laborious task of reconstructing the new K book, when both the book and the tables were captured from Geier, a German trawler, on 26 December 1941.
Hut 8 had been aware since early 1941 that there was a four-rotor naval Enigma – M4. In September 1941, it learned that some U-boats had been issued with the new machine, when an M4 lid was recovered from U-570, which had surrendered to an aircraft on 27 August. It was therefore far from a complete surprise when M4 came into operation on Shark on 1 February 1942. Hut 8 had in fact already solved the wiring of the new rotor, beta, and its associated thin reflector, Bruno, in December 1941, after M4 was used accidentally on several occasions. But the combination of M4, a separate cipher (Shark) and the introduction of a second edition of the Wetterkurzschlüssel on 20 January 1942 were calamitous for Hut 8. Deprived of cribs and without four-rotor bombes, it became blind against Shark, solving only three Shark keys during the next ten months.
The Bletchley Park Codebreakers Page 21