by Alastair Denniston- Code-breaking From Room 40 to Berkeley Street
AEF American Expeditionary Force
AI Air Intelligence (Directorate)
ASA Army Security Agency (US)
ATB Advisory Committee on Trade Questions in Time of War
BEF British Expeditionary Force
BJs ‘Blue jackets’ (nickname for GC&CS reports)
BOT Board of Trade
BP Bletchley Park
BSC British Security Coordination
‘C’ Chief of the Secret Service
Cab. Cons. Cabinet Conclusions (minutes of Cabinet meetings)
CAS Chief of the Air Staff
CBME Combined Bureau Middle East
CID Committee of Imperial Defence
CIGS Chief of the Imperial General Staff
COMINT Communications intelligence
COMINTERN Communist International organisation
COMSEC Communications security
CNS Chief of the Naval Staff
COS Chiefs of Staff
CSS Chief of the Secret Service
CTO Central Telegraph Office
D & R Distribution and Reference (Office)
DCAS Deputy Chief of the Air Staff
DCIGS Deputy Chief of the Imperial General Staff
DCOS Deputy Chiefs of Staff
DD(C) Deputy Director (Civil)
DDI Deputy Director of Intelligence (Air Ministry)
DDMI Deputy Director, Military Intelligence
DD(S) Deputy Director (Service)
DID Director of Intelligence Division (Admiralty)
DMI Director of Military Intelligence
DNI Director of Naval Intelligence
DOD Director Operations Division (Admiralty)
DRC Defence Requirements Committee
DSLTB Defence Services Lines Telecommunications Board
D/F Direction finding
ELINT Electronics intelligence
FECB Far East Combined Bureau
GAF German Air Force
GC&CS Government Code and Cypher School
GCHQ Government Communications Headquarters
GHQ General Headquarters
GPO General Post Office
HVB Handelsverkehrsbuch
IE Intelligence Exchange
ISK Intelligence Services, Knox
ISOS Intelligence Services, Oliver Strachey
ISSIS Interservice Special Intelligence School
JCC Joint Committee of Control
JIC Joint Intelligence Sub-committee (of the COS)
JIG Japanese Imperial Government
JPC, JP Joint Planning Sub-committee (of the COS)
MEW Ministry of Economic Warfare
MI Military Intelligence (Directorate)
MI3 German Division, Military Intelligence Directorate
MI5 Security Service
MI6 Secret Intelligence Service (SIS)
MSS ‘Most Secret Source’ – original designation for ‘Ultra’ material
NID Naval Intelligence Division
NSA National Security Agency
OD Operations Division (Admiralty)
OIC Operational Intelligence Centre (Admiralty)
OTP One-Time Pad
PWE Political Warfare Executive
RN Royal Navy
RNVR Royal Naval Volunteer Reserve
RSS Radio Security Service
Sigint Signals intelligence
SIS Secret Intelligence Service (MI6) (British)/Signal Intelligence Service (US)
SKM Signalbuch der Kaiserlichen Marine
SOE Special Operations Executive
SSA Signal Security Agency (US – formerly SIS)
TNA The National Archive, Kew, London
VB Verkehrsbuch
WO War Office
WOG Wireless Observation Group (Army)
WW1 World War One
WW2 World War Two
W/T Wireless telegraphy
WTI W/T intelligence
Chapter 1
A Life in Signals Intelligence
The ability to acquire and apply knowledge and skills, commonly known as intelligence, is invaluable in all walks of life. It is of particular value in political and military affairs. Military intelligence on its own does not win or shorten wars but it does help to shape their course and in unusual ways. It provides information which may improve the ‘number of chances’ at a commander’s disposal.1 E.T. Williams, head of intelligence for the 21st Army Group during WW2 noted that:
Perfect Intelligence in war must of necessity be out-of-date and therefore cease to be perfect. We deal with partial and outmoded sources from which we attempt to compose an intelligible appreciation having regard to the rules of evidence and our soldierly training and which we must be prepared constantly to revise as new evidence merges. We deal not with the true but with the likely. Speed is therefore of the essence of the matter.2
Intelligence, in a military sense, aims to minimise uncertainty about the enemy and at the same time, maximise the efficient use of one’s own resources. So the challenge for intelligence services is to provide current, reliable and effective information which commanders in the field can add to their own knowledge. This information typically emerges from an intelligence chain, which starts with the interception of the enemy’s communications or signals. This is followed in turn by decryption, translation, analysis, distribution and action. This process can be flawed at any point in the chain and pieces of intelligence are usually fragmentary and controversial, with their value never entirely clear. They can generate debate as much as information, and inferences drawn from a number of messages have to be combined into an assessment which can then generate a strategy for action. In the end, its effectiveness depends on a military organisation able to exploit it.
During WW1, the intelligence branch at General Headquarters (GHQ) in Iraq noted that ‘the mass of information received by GHQ almost always came in in disconnected fragments of very varying value, fragments were the general rule, and the bulk of intelligence work at any GHQ really consists in putting together a gigantic jig-saw puzzle from innumerable little bits, good, bad and indifferent.’3 Commanders wished to gather information on all aspects of the enemy. However, their most fundamental needs were to determine the intentions and movements of their enemy. Walter Kirke, a senior member of the intelligence branch at GHQ in France during WW,1 termed this ‘the bed-rock of all intelligence work’, to uncover the enemy’s order of battle, that is, the locations, strength and organisation of its troops.4
William Frederick Friedman was arguably the leading American expert in the field of signals intelligence during the first half of the twentieth century. In a series of lectures, serialised in the NSA Technical Journal from 1959 to 1961, Friedman attempted to define the various components of signals intelligence and its historical origins.5 In his view, signals intelligence (Sigint) has two main components: communications intelligence (COMINT) – information derived from the organised interception, study and analysis of the enemy’s communications, and electronic intelligence (ELINT) – information obtained from a study of enemy electronic emissions such as homing or directional beacons, radar, recording data of an electronic nature at a distance. Not only do most countries seek to obtain Sigint even during peacetime, most also invest heavily in protecting their own communications. Friedman defined this latter activity as communications security (COMSEC) – the protection resulting from all measures designed to deny to the enemy information of value that may be derived from the interception and study of such communications. A few other definitions are needed at this time: cryptology is the science that is concerned with all of these branches of secret signalling, cryptography is the science of preparing secret communications and cryptanalysis is the science of solving secret communications. So in summary, Sigint involves the interception of messages, traffic analysis – the study of unencrypted information contained within the messages such as the identity of the sender, recipient, etc. – and the solution of codes and ciphers.
All messages sent through
a communications system start off life in so-called plaintext, usually the language of the sender. They are converted or transformed by following certain rules, steps or processes to disguise them. For the purposes of this book, the conversion or transformation is called encryption and the reverse process, decryption. In general terms, the resulting disguised text can be called a cryptogram. This terminology is often confusing for new readers because some authors prefer to use encode or encipher instead of encrypt and decode or decipher instead of decrypt. The reason for this is that encrypting and decrypting are achieved by means of codes and ciphers which lay at the heart of Sigint and COMSEC systems. It is important to understand the difference between the two. In cipher or cipher systems, cryptograms are produced by applying specific rules, steps or processes to individual letters of the plaintext. These types of cryptograms can be said to be in cipher text. In code or code systems, cryptograms are produced by applying specific rules, steps or processes generally to entire words, phrases and sentences of the plaintext.
The earliest reliable information of the use of cryptography in connection with an alphabetic language dates from about 900 BC. Use was made of a device called a scytale, a wooden cylinder of specific dimensions around which was wrapped spirally a piece of parchment or leather. The message was written on the parchment, unwound and sent to its destination by a safe courier. The recipient would have the same device to wind the parchment on, thus bringing together properly the letters representing the message. However, exact details of how this device worked in practice are unknown.
There are number of examples of ciphers in the Bible and one of the more interesting ones involves the mention in Jeremiah 25:26 and Jeremiah 51:41 of a place called Sheshakh. This was unknown to geographers and historians, until a coding system was discovered using the Hebrew language. If you write the twenty-two letters of the Hebrew alphabet in two rows, letters 11 to 1 in one and letters 12 to 22 in another, you have a substitution alphabet where you can replace letters with those opposite. This is called ATHBASH writing where aleph, the first letter, is replaced by tech, the last letter; beth, the second letter by shin, the next-to-last, etc. This revealed that Sheshakh actually translates as Babel, the ancient name for Babylon.
However, the world of signals intelligence that Alastair Denniston would serve for thirty years really began in 1653. The first regular interception and cryptanalytic organisation in Britain was born with the establishment of the postal monopoly and the Secret Office in 1653. In 1657, Parliament passed the first Post Office Act and the separate Inland and Continental postal offices were united into a General Post Office. This provided a legal basis under which the power of the Secretary of State to issue warrants was recognised, and authorised the Postmaster General to open and examine correspondence. Postal rates were fixed and John Thurloe6 became the first Postmaster General. Throughout the eighteenth century, the Post Office transmitted, collected and created intelligence. This was achieved by the simple expediency of opening, detaining and copying correspondence and then sending it on to the Secretaries of State. The Post Office Act of 1711 guaranteed a regular source of material from the Post Office’s monopoly of the mails, supplemented by occasional captures of documents in war time, or from the activities of secret agents.
Inland post was examined in the Private Office while foreign post was examined in a special office known as the Secret Office. The Private Office of the Secretary of the Post Office was responsible for the execution of warrants to intercept inland mail in connection with political and criminal investigations. The Secret Office was tasked with opening, reading, copying and re-sealing letters and dispatches, and sometimes deciphering those in simpler codes or ciphers. It was located in Post Office premises but was responsible to the appropriate Secretary of State. Eventually, it dealt almost exclusively with foreign mail and was responsible to the Secretary of State for the Foreign Department. It had no official existence and was headed by the Foreign Secretary. Its first manager was Isaac Dorislaus,7 who was known intriguingly as the Secret Man. The Foreign Secretary was responsible for supervising the opening and copying of foreign correspondence and sending it on to the Secretaries of State in packets marked ‘Private and Most Secret’. Those in plain text were sent directly to the King while those in cipher were passed on to another department known as the Deciphering Branch.
The Deciphering Branch was responsible for cryptography and translation and, from 1762, also undertook experimental work. It had no specific location, formal organisation or head. However, one of its most successful operatives was Dr John Wallis, a famous mathematician who can lay claim to being the father of British cryptography.8 Wallis’ assistant was his grandson, William Blencowe, an undergraduate at Magdalen College, Oxford. On Wallis’ death in 1703, Blencowe became the first official Decipherer. The branch was staffed by two or three experts working on their own as research specialists to investigate new ciphers and, if possible, solve them. From a staff of around five or six in the early years, it eventually grew to about twelve in number.
There are numerous examples of the Post Office’s intelligence work helping to guarantee the safety of the British Empire. One example was intelligence which warned the British forces at Philadelphia of the arrival of the French fleet in 1778 during the American Revolutionary War. Foreign correspondence from many countries was read, including that of the courts of France, Prussia, Austria, Russia, Spain, Sardinia, Holland and Sweden, and intercepts averaged two or three per week. The Hanoverian government which ruled the United Kingdom from 1714 to 1837 maintained a ‘secret bureau’ of openers and deciphers at Neinburg (a district in Lower Saxony, Germany). Interceptions were sometimes obtained from agents or foreign postmasters in Brussels, Danzig, Hamburg, Leyden and Rotterdam. The security of the Post Office’s intelligence operation depended on the skill of its operatives, restricted knowledge, loyalty and the absence of Parliamentary criticism. The distinction between the Private Office, Secret Office and Deciphering Branch, along with a short distribution list of intelligence, helped to maintain secrecy. A system of recruitment and training was put in place, based on patronage and nepotism. This served to provide suitably motivated and reliable individuals from a small number of family dynasties for both the Secret Office and the Deciphering Branch.
In June 1844, the Government faced criticism as some of its intelligence activities became known. At the end of June, it stopped the interception of diplomatic correspondence in the Secret Office. This was done on the basis that the act of 1711 only authorised express as opposed to general warrants for the opening of post. However, the Home Secretary of the day, Sir James Graham, had signed a warrant for the interception of the correspondence of Giuseppe Mazzini. He was an Italian nationalist living in Britain and the Austrian Ambassador was concerned that he was conducting subversive activities. Mazzini’s letters were intercepted between March and June 1844 and sent to the Foreign Secretary. Mazzini discovered that his letters had been intercepted and complained to a Member of Parliament. The issuing of general warrants could not be defended by the Government and in February 1845, the Home Secretary announced the department formally maintained in the Post Office by the Secretary of State for the Foreign Office had been abolished and that no similar establishment was maintained by the Home Office. As public opinion against such activity mounted, in August the Government abolished the Secret Office and in October, the Deciphering Branch. The last official Decipherer was Francis Willes, who worked alone apart from an assistant from 1825 until the branch was abolished. The Private Office gradually became less important, as the attitude of the day was that England ‘does not stand in need of such expedients for her safety’.9 It was abolished along with the Deciphering Branch in 1844.
Under the Hanoverians (from George I in 1714 to the end of Victoria’s reign in 1901), British Sigint provided technical assistance in opening and re-sealing dispatches, provision of staff, and a service of deciphered product from the King’s Hanoverian Sigint organi
sation. It had considerable success in solving the diplomatic ciphers of many European countries and provided a service of decrypts of the diplomatic dispatches for the King and his most senior ministers only, providing support to managing affairs of state and the conduct of diplomacy.
It does seem that the British Sigint organisation was broken up and abandoned in 1844, and the expertise lost, much to the detriment of British cipher security. While telegrams could legally be intercepted, this could be done only by individual warrants to investigate political or criminal matters. Remarkably, most Victorian statesmen maintained strict moral codes of gentlemanly conduct and there appears to be no evidence in either GCHQ or other archives of British Sigint activity between 1844 and 1914.
***
The telegraph was first developed by Samuel F.B. Morse, an artistturned-inventor who conceived the idea of the electric telegraph in 1832. Several European inventors had proposed such a device, but Morse, working independently, had by the mid-1830s built a working telegraph instrument. In the late 1830s, he perfected Morse Code, a set of signals that could represent language in telegraph messages. In May 1844, Morse inaugurated the world’s first commercial telegraph line with the message ‘What hath God wrought’, sent from the US Capitol to a railroad station in Baltimore. Within a decade, more than 20,000 miles of telegraph cable criss-crossed the country. The rapid communication it made possible greatly aided American expansion, making railroad travel safer as it provided a boost to business conducted across the great distances of a growing United States.
The idea of a transatlantic communications cable was first raised in 1839, following the introduction of the working telegraph by William Fothergill Cooke and Charles Wheatstone. Morse threw his weight behind it in 1840, and by 1850, a link had been laid between Britain and France. The same year, construction began on a telegraph line up the far north-east coast of North America – from Nova Scotia to the very tip of Newfoundland. Cyrus West Field, a businessman and financier from New York City, took up the idea of extending the east-coast cable across the Atlantic to Britain. In 1857, after several attempts had failed, two ships, the USS Niagara and HMS Agamemnon, met in the centre of the Atlantic on 29 July 1858, and attached the cables together. This time there were no cable breaks, and the Niagara made it to Trinity Bay in Newfoundland on 4 August, and the Agamemnon arrived at Valentia Island off the west coast of Ireland on 5 August. Over the following days, the shore ends were landed on both sides using a team of horses, and tests were conducted. On 16 August 1858, the first message was sent across the Atlantic by telegraph cable, reading ‘Glory to God in the highest; on earth, peace and good will toward men’. The transmission marked the culmination of nineteen years of dreams, plans and hard work, bridging the economic and political systems of both the UK and the US. The reception across the cable was terrible, and it took an average of two minutes and five seconds to transmit a single character. The first message took 17 hours and 40 minutes to transmit. On 3 September 1858, the cable failed. In an attempt to increase the speed of transmission, the voltage on the line was boosted from 600V to 2,000V, and the insulation on the cable couldn’t cope. It failed over the course of a few hours, and it would be another six years before the capital was raised for another attempt. By the end of the nineteenth century, the British Empire was self-sufficient in its cable infrastructure. All parts of the Empire could be reached with British-owned cable with sufficient redundancy built in.