Hut 6 encountered a different change to Enigma on 10 July 1944 when, without any warning, some Luftwaffe Enigma decrypts yielded only a number, which was later recognized as an indicator, followed by nonsense. It took Hut 6 about four days from the receipt of the first of the messages to work out the wiring: it realized that a substitution device (called the Uhr by the Germans) had been employed between the clear-text and the rotors. Forty such Stecker substitutions, of which thirty were non-reciprocal, were eventually isolated and their interrelationship discovered, which then allowed them all to be deduced after one was broken. The Uhr, which was later applied to about fifteen Luftwaffe ciphers, only had a slight effect on breaking Enigma, since the bombes could still be used, although the diagonal board was rendered useless by the non-reciprocal Stecker. The Uhr was a nuisance – and not a significant problem – for Hut 6, although interestingly the Germans attached much more importance to it than to UKD, probably because they placed undue reliance on Enigma’s plugboard.
The Germans had designed a cipher machine, Schlüsselgerät 39 (cipher machine 39 – SG 39), in 1939 to replace Enigma, but it never went into mass production. The SG 39 was an outstanding machine, embodying a rewirable reflector, four variable notch rotors, three mechanical drive wheels, and an entry plugboard. The Allies could not have broken it. The Germans developed a variable-notch rotor (Lückenfüllerwalze) for Enigma as an interim measure, pending the introduction of the SG 39. Plans were made in December 1943 to bring one such rotor into service with Heer Enigmas ‘within a reasonable time’, since replacing all the existing rotors would have been a huge logistical task. Although it was a very simple device, the new rotor would almost certainly have foiled the bombes, by making it impossible to know when the middle and right-hand rotors advanced.
OKW/Chi’s Oberinspektor Fritz Menzer, who was a thorn in the flesh of the ISK Section (Abwehr machine ciphers) at Bletchley, developed a Hagelin-type machine, the Schlüsselgerät 41 (cipher machine 41 – SG 41). About 11,000 machines were ordered by the Heer and Luftwaffe, but few were produced. When it was brought into service with the Abwehr in late 1944, GC&CS solved a few messages using it by aligning them in depth, but could not reconstruct the machine itself.
However, UKD had posed the greatest potential threat to Ultra. A postwar United States Army study concluded that:
only a trickle of solutions would have resulted if [UKD] had been adopted universally; and this trickle of solutions would not have contained enough intelligence to furnish the data for cribs needed in subsequent solutions. Thus even the trickle would have eventually vanished.
The Allies were very fortunate indeed that Ultra from Enigma did not dry up long before the end of the war, especially since, with better management, the Wehrmacht could have greatly increased Enigma’s security without undue difficulty.
21
FROM AMATEURS TO PROFESSIONALS: GC&CS AND INSTITUTION-BUILDING IN SIGNALS IN TELLIGENCE
PHILIP H. J. DAVIES
Introduction
Throughout the interwar years GC&CS was a remarkably relaxed organization. It recruited only from within a certain circle of people, virtually everyone who joined had been recommended by a friend of one of the codebreakers. This was deemed to be the only way of ensuring that the codebreakers’ work remained secret but, as Josh Cooper recalled, it was not conducive to good organization. ‘Recruitment by personal introduction had produced a number of very well-connected officers. At best, they were fine scholar linguists, at worst some of them were, frankly, passengers.’
Alastair Denniston had virtually no resources with which to improve the situation and it was not until 1937 when, with the backing of Admiral Hugh Sinclair, he toured the universities looking for mathematicians and linguists to work on the Enigma ciphers, that the situation began to change. This showed far more prescience than either man has been credited with. ‘It would be hard to exaggerate the importance of this course for the future development of GC&CS,’ Josh Cooper said. ‘Not only had Denniston brought in scholars of the humanities, of the type of many of his own permanent staff, but he had also invited mathematicians of a somewhat different type who were specially attracted by the Enigma problem. I have heard some cynics on the permanent staff scoffing at this. They did not realize that Denniston, for all his diminutive stature, was a bigger man than they.’ The new recruits included men like Gordon Welchman and Hugh Alexander. They began to take an interest in the organizational structure of Bletchley Park and introduced measures to improve the process of intelligence production, something that had been anathema to many of the interwar codebreakers, who were only concerned with the solution of the ciphers, not with passing on the intelligence they produced.
This chapter describes the subsequent evolution of GC&CS into the postwar Government Communications Headquarters (GCHQ) and the internal and external pressures that led to the transformation from a small group of ‘Gentlemen Codebreakers’ to the large-scale, highly organized postwar Sigint organization GCHQ.
MS
It is difficult to really understand an organization like GC&CS outside of its contemporary institutional context of Britain’s wartime intelligence community, and its longer term historical context as a phase in the evolution of today’s Government Communications Headquarters (GCHQ). To appreciate how GC&CS evolved, why it evolved the way it did, and what that evolution means, it is valuable to compare GC&CS and its sister intelligence and security services, the Secret Intelligence Service (now better known as MI6), and the Security Service (MI5).
Although GCHQ, as a Sigint organization, collects information from a fundamentally different class of sources from MI6 (which concentrates on human agents and local technical operations, usually in conjunction with GCHQ), MI5 has to handle both communications as well as human intelligence domestically (again in concert with GCHQ). However, like MI6, GCHQ is essentially a service provider to consumers in the overt government and as a result is subject to very similar structural and political pressures. By contrast, MI5, which is supposed to be as independent from political interference as possible under both the 1952 Maxwell-Fyffe Directive (which charged it with ‘the Defence of the Realm’) and the 1989 and 1996 Security Service Acts, does not really serve a brace of intelligence ‘consumers’ in quite the same way. As a result, GCHQ, like MI6, evolved very much on the basis of consumer demand for a particular kind of intelligence. The pressures of that demand shaped its early loose and informal structure as the interwar GC&CS, and also proved central to the pressures and crises during its most productive years, churning out Ultra during the Second World War. It was out of that period of pressure and transformation that much of what constitutes the modern GCHQ took shape.
Compared with both MI6 and GCHQ, MI5 developed throughout its history as a fairly coherent, hierarchical bureaucracy. In part, one might trace the tighter organizational practice at the Security Service back to the earliest days of Mansfield Cumming and Vernon Kell, founding fathers of the foreign and domestic sides of the Secret Service Bureau. Those two sides would eventually split apart to become MI6 and MI5. Even from the earliest days, Kell, the War Office official, displayed a greater flair for organizational design than the ageing and retired naval commander Cumming. However, such an early influence need not prove pervasive and lasting, and it is fairly easy to show that, regardless of flair, MI6 was as subject to the range of pressures within Whitehall that prevented it from adopting a conventional hierarchy.
The strongest single pressure on MI6 was consumer demand. The most important thing to understand about consumer demand in terms of Britain’s intelligence machinery is that intelligence in the broad sense, that is, any and all information relevant to policy formation and execution, is a departmental concern, and not an agency one. Intelligence, at least in the sense of special or secret intelligence, is essentially one more of a variety of information sources which the overt government can employ in its ordinary course of work. According to Reginald Hibbert, a government official,
around 50 per cent of the information used by the Foreign Office is ‘drawn from overt published sources’. A further 10 to 20 per cent is ‘privileged material which is not strictly speaking classified’ or information which is ‘classifiable as confidential’ and is ‘a product of normal diplomatic activity’, while of the remainder only around 10 per cent comes from secret sources. Traditionally, the various forms of covert intelligence collection were set up as required more or less independently by the various departments. Hence, during the Napoleonic Wars military intelligence networks were set up, but dismantled after the conflict, just as the Indian Government employed the players of the Great Game on an ad hoc basis. Even the long-standing Secret Department of the Post Office had been shut down in 1840. It was not until after the Boer War that there was any effort to set up a permanent secret service, and even that was a very limited undertaking. When the first really permanent secret service was set up in 1909, it was primarily because of concerns expressed by the Admiralty and War Office before the Haldane Committee ‘into the question of foreign espionage’ that they lacked information about German ports and dockyards, and the ability to deal securely with potential agents.
The importance of customer demand is well illustrated by the fact that, in 1917, Cumming was forced to reorganize his headquarters from a geographical structure to one based on the functional relevance to his consumer departments. Under the so-called Macdonogh Scheme, the MI6 HQ was then divided into sections that provided military intelligence for the War Office, economic intelligence for the Ministry of Blockade and political intelligence for the Foreign Office and so forth. This reform, imposed on Cumming by his Whitehall masters, meant two main things. In the first place, it demonstrated that he was not really master in his own home. What the customers in Whitehall and Downing Street wanted would prevail over any internal process. It also flattened the MI6 headquarters structure so that it acted mainly as a distributor of information to those consumers where the first responsibility of the collating sections was to their customers rather than Cumming or his successors as Chief or ‘C’. This dynamic of consumer demand was intensified after the First World War by the decision in 1921 to give MI6 exclusive responsibility for collecting intelligence for all three services and a number of government departments. As a quid pro quo for MI6’s monopoly, the principal ‘consumers’ attached sections of their own intelligence branches to MI6 HQ to act as collators, as conduits of requirements (or ‘tasking’) issued to MI6 by their home departments, and as circulators of intelligence reports from MI6 to their departments. These sections were dubbed Circulating Sections.
The doctrine of ‘dual control’ (as it would later become known in MI6), originally embodied in the Circulating Sections, meant that senior headquarters officers owed only part loyalty to ‘C’, and the rest to the War Office, Admiralty or wherever they had come from. This created a precedent of loose and horizontal organization which carried over into the addition of other sections for internal ciphers and radio communications, escape and evasion and so forth. This made the service desperately difficult to manage, especially when the Second World War broke out and staff numbers and additional specialist sections began to multiply. As a consequence, one-time Deputy Chief Valentine Vivian eventually described the service as a loosely affiliated group of ‘independent sections … known as the SIS’. This weak structure made the service almost impossible to oversee and direct by the middle of the war, and successive attempts at reform eventually led to a postwar scheme in which the service was centrally divided between a ‘Production side’, which collected information on a geographical basis, and a ‘Requirements side’, which collated and disseminated it and received consumer requirements, on a functional basis.
By comparison, MI5 has always existed with a certain degree of detachment from the rest of the British machinery of government. Production and dissemination of raw intelligence to consumers in Whitehall and Downing Street were never its main function. Moreover, during its interwar and early wartime years, MI5 was very much without a specific governmental master or ministerial sponsor. Indeed, the Security Service’s main consumer is, and has always been, the service itself. As a result, it has never been subject to the same kind of tangential forces that pulled MI6 into a wide, flat, ‘organic’ structure. On the contrary, the problems experienced by MI5 have generally had to do with excessively rigid bureaucracy rather than overly loose organization. However, MI5 would experience problems of rapid growth and unanticipated success during the Second World War, much as GC&CS did, the crucial difference in their experiences being the fact that the latter’s position in the machinery of government was (and in its modern form as GCH Q is) more akin to that of MI6 than MI5.
Like MI6, the key force in the evolution of GC&CS was consumer demand, and the desire of consumers to ensure access to the service’s product. Its origins lay in an attempt to regularize the cryptanalytic machinery which had developed during the war, chiefly in the form of the Admiralty’s highly successful ‘Room 40’ staff, and the less dramatic efforts of the War Office cryptanalysis section MI1b. Just as early human intelligence had been run on a departmental, ad hoc basis, so wartime codebreaking had been chiefly a departmental concern. This was so much the case that considerable rivalry and hostility existed between the Admiralty and War Office codebreaking teams, undermining the potential advantages to be had from pooling their efforts. Regardless of the risks to security from weakening compartmentalization by co-operating, codebreaking is a field with very real economies of scale where different groups working on different problems can generate results useful to each other as well as to themselves. This was certainly the experience of the First World War, and it was because of this that GC&CS was established on an inter-departmental basis in 1919. That being said, in the first instance, GC&CS came under the authority of the Foreign Office, and under ‘C’ when Admiral Sir Hugh ‘Quex’ Sinclair took over that post after the death of the MI6 founder Mansfield Cumming. The original GC&CS consisted of its director, six senior assistants, drawn from the best cryptanalysts of Room 40 and MI1b, eighteen junior assistants and a clerical staff of twenty-eight.
GC&CS was housed with MI6 at 54 Broadway, near Whitehall, where they were known collectively and euphemistically as the Government Communications Bureau (GCB). The original organization was very loosely and informally organized, partly a carry-over of the organizational culture of Room 40 and partly a consequence of its relatively small size. Formal management structure took a back-seat to getting the job done, so much so that one historian has summarized the experience of the interwar GC&CS as a combination of ‘technical success and organizational confusion’.
The resulting organization, however, like MI6, was to become profoundly shaped by the role and interests of its consumers as it developed. Rather than being confined to its original single, central cryptanalysis staff under its interwar chief, Alastair Denniston, its original postwar staff was added to in the form of service branch cryptanalytical sections and a Foreign Office-oriented civilian side, which were attached on terms of dual control similar to the circulating sections of MI6. Almost as soon as GC&CS was set up, the service branches began to complain that service cryptanalytical interests and needs were not being properly addressed. A Naval Section was formed in 1924, an Army Section in 1930, and an Air Section in 1936. In 1938, a commercial section was set up to provide intelligence to the Industrial Intelligence Centre (IIC) at the Department of Overseas Trade. From 1937, the ‘civil side’ included a GPO-manned system of stations intercepting Axis diplomatic traffic for Foreign Office consumption, although GC&CS had had access to pretty much any telegram traffic required from Britain’s commercial cable companies throughout the interwar period. Once war broke out, there was even a meteorological section attacking enciphered German weather reports.
It might be tempting to suggest that part of the reason for this weak internal structure was a carry-over from MI6. This would be a substantial misreading of the situat
ion as the two organizations had only a minimum of interaction, the main links being via ‘C’, who oversaw both, and the MI6 circulating sections which tasked the operational sides of both services. The main factor driving the matrix of dual-control sections that took shape as GC&CS was the individual departmental contributions of staff and resources to the service, and the desire of those departments to make use of its product. However, the internal processes were further complicated by the fact that the MI6 circulating sections also acted as conduits for consumer requirements to GC&CS during the interwar years.
If the demand for intelligence is a crucial input to intelligence service structure in the British model, the lack of demand can be equally telling. Although the War Office, Admiralty and Air Force all contributed to the make-up of GC&CS, this did not make it a highly valued or well-endowed organization. Throughout the interwar period, it suffered the same financial stringency as the other intelligence services and the armed forces under ‘the Ten Year Rule’ – the assumption that there would be no major war for at least a decade. The result was an acute lack of receivers for intercepting foreign traffic, and therefore a dearth of traffic to study in order to achieve any significant breaks. This problem was intensified by the fact that most continental armed forces adopted relatively fixed, stationary deployments during the inter-war period, and so secure communications moved mainly to landlines leaving relatively trivial and generally unencoded traffic to go by radio.
During mid-1940, Air Intelligence did attach a small traffic analysis team to Bletchley Park’s Air Section. This was considerably enhanced when the War Office intercept organization MI8 attached all its traffic analysts to Bletchley as well. Denniston originally resisted this, insisting that GC&CS should remain concerned strictly with cryptanalysis (although he would later insist on a consolidation of all Sigint work in the Middle East theatre). Apart from any issue of the advantages or disadvantages of consolidating Sigint at Bletchley, one factor almost certainly in Denniston’s mind was the increasingly crowded circumstances at Bletchley Park. There was, however, external pressure for an amalgamation of Sigint. The Director of Military Intelligence wrote to his fellow service directors of intelligence lamenting the lack of co-ordination between traffic analysis and cryptanalysis, and the overall direction of Sigint. He demanded an investigation of the body responsible for the overall handling of intercept work, the Y Committee, and even suggested subordinating it to the Joint Intelligence Committee (JIC).
The Bletchley Park Codebreakers Page 42