My heartfelt thanks go to Colonel William J. Williams, USAF (ret.), and his staff at the National Security Agency’s Center for Cryptologic History (CCH). The work of the CCH historians runs throughout this history. It is fair to say that this book would not have been possible without them.
And finally, I would also like to extend his most heartfelt thanks to the staff of the National Archives at College Park, Mary land, for helping me conduct my research over the past two decades. I will always remain deeply indebted to the late John E. Taylor, the doyen of military archivists at the National Archives, whose encyclopedic knowledge of the records based on his fifty years at the archives was unparalleled anywhere. His passing in September 2008 at the age of eighty-seven marks the end of an era. The staff of the NARA Library at College Park, especially its amiable head Jeff Hartley, helped me work the CIA’s CREST database of declassified documents through many trials and tribulations, and stoically processed the vast amount of declassified documents that I brought to their desks day after day without complaint. They are wonderful people.
My deepest gratitude goes to Peter Ginna, my publisher at Bloomsbury Press, who to his eternal credit took a risk and agreed to publish this book. Michael O’Connor and Pete Beatty did the heavy lifting at Bloomsbury getting this opus ready for publication. Special thanks go to my editor James O. Wade, who performed a Herculean effort to get this manuscript into final form. And last but not least, my agent, Rick Broadhead, worked tirelessly on this project, believing implicitly in the importance of what I was trying to accomplish.
Notes Glossary
AIA Air Intelligence Agency
ASA Army Security Agency
CALL Center for Army Lessons Learned
CCH Center for Cryptologic History, Fort George G. Meade, Maryland
CNSG Crane Naval Security Group Archives
DCI Director of Central Intelligence
DDEL Dwight D. Eisenhower Library, Abilene, Kansas
DDRS Declassified Document Retrieval Service
DOCID Document Identification number
DOD Department of Defense
FBI Federal Bureau of Investigations
FOIA Obtained by Freedom of Information Act request
GPO Government Printing Office
HCC Historic Cryptologic Collection, contained in Record Group 457 at the National Archives, College Park, Maryland
HSTL Harry S. Truman Library, Independence, Missouri
INR State Department, Bureau of Intelligence and Research
INSCOM U.S. Army Intelligence and Security Command
JCS Joint Chiefs of Staff
JFKL John F. Kennedy Library, Boston, Massachusetts
LBJL Lyndon Baines Johnson Library, Austin, Texas
NA, CP National Archives, College Park, Maryland
NARA National Archives and Records Administration, Washington, D.C.
NIO IIM National Intelligence Officer Interagency Intelligence Memorandum
NSA OH NSA Oral History, held by the NSA’s Center for Cryptologic History, Fort George G. Meade, Maryland, and obtained through FOIA
PRO Public Records Office, now National Archives of the United Kingdom, Kew, England
RG- Record Group
RUMRA NSA internal designation for the Russian communications target: “RU” = Russia; “M” = Army; “RA” = mainline Morse code circuit
SSA Signal Security Agency
Notes
Prologue
1. Background and character of Clarke from U.S. Army biographical data sheet, Brigadier General Carter Weldon Clarke, USA (Ret.); interviews with W. Preston Corderman, Frank B. Rowlett, Morton A. Rubin; NSA, oral history, Interview with Carter W. Clarke, May 3, 1983; NSA OH-01-74 to NSA OH-14-81, oral history, Interview with Frank B. Rowlett, 1976, p. 33, NSA FOIA. See also memorandum, Ohly to McNarney, Your Proposals with Respect to the Handling of Communications Intelligence and Communications Security, May 12, 1949, p. 1, RG-330, entry 199, box 97, file: CD 22-1-23, NA, CP; Henry C. Clausen and Bruce Lee, Pearl Harbor: Final Judgement (New York: Crown, 1992), p. 24.
2. NSA OH-01-74 to NSA OH-14-81, oral history, Interview with Frank B. Rowlett, June 26, 1974, p. 76, NSA FOIA.
3. For the genesis of the SIGINT effort against the USSR in 1943, see Robert Louis Benson and Cecil Phillips, History of VENONA (Fort Meade, MD: Center for Cryptologic History, 1995), vol. 1, p. 12, NSA FOIA; Robert Louis Benson and Michael Warner, eds., VENONA: Soviet Espionage and the American Response, 1939–1957 (Washington, DC: Center for the Study of Intelligence, 1996), p. xiii. For the intense secrecy surrounding the Russian code-breaking effort, see memorandum, Corderman to Taylor, Draft of “Priorities Schedule,” March 6, 1943, and memorandum, Taylor to Corderman, SPSIS 311.5—General—Draft of “Priorities Schedule,” March 8, 1943, both in RG-457, HCC, box 1432, file: SSS Intercept Priorities, NA, CP; Benson and Phillips, History of VENONA, vol. 1, p. 16 and fn27. For the U.S. Navy’s parallel SIGINT effort against the Soviet Union, see Naval Communications Activity, Russian Language Section: July 1943–January 1948, NSA FOIA via Dr. David Alvarez; Dr. Thomas R. Johnson, American Cryptology During the Cold War, 1945–1989 (Fort Meade, MD: Center for Cryptologic History, 1995), bk. 1, The Struggle for Centralization, 1945–1960, p. 159, NSA FOIA. For the problematic cooperation between the army and navy on the Russian problem, see Thomas L. Burns, The Origins of the National Security Agency: 1940–1952 (Fort Meade, MD: Center for Cryptologic History, 1990), p. 25, NSA FOIA.
4. SRH-364, History of the Signal Security Agency, 1939–1945, vol. 1, p. 139ff, RG-457, entry 9002 Special Research Histories, NA, CP; “Hot Weather Policy,” NSA Newsletter, June 1956, p. 23; Debbie DuBois, “Those Good Old Days,” NSA Newsletter, December 1979, p. 7; Jack Gurin, “Dear Old Arlington Hall,” NSA Newsletter, February 1981, p. 14, all NSA FOIA; “From Coeds to Codewords: How a Girls College Became the Nerve Center for USASA’s Global Operations,” Hallmark, September 1970, p. 8, INSCOM FOIA; “Forty One and Strong: Arlington Hall Station,” INSCOM Journal, June 1983, pp. 7–12, INSCOM FOIA; U.S. Army Intelligence and Security Command, INSCOM and Its Heritage (Arlington, VA: INSCOM History Office, 1985), Special Historical Series, pp. 137–38, INSCOM FOIA.
5. For keeping the SIGINT effort against the Soviets a secret from the British, see Benson and Phillips, History of VENONA, vol. 1, p. 16 and fn27. For details of the British code-breaking effort against the USSR during World War II, including the fact that this operation was kept secret from the United States, see Benson and Phillips, History of VENONA, vol. 1, pp. 30–31; Burns, Origins of the National Security Agency, p. 25; NSA OH-01-79, oral history, Interview with Brigadier John H. Tiltman (ret.), January 30, 1979, p. 1, NSA FOIA; NSA OH-20-93, oral history, Interview with Oliver R. Kirby, June 11, 1993, pp. 10–11, NSA FOIA; handwritten notes labeled “CDR Dunderdale,” undated, in OP-20-G organizational file, NSA FOIA.
6. Hallock was one of the first men to excavate the old capital of the Achaemenid civilization at Perse-polis in Iran. Recruited into the Signal Security Agency in 1942 because of his linguistic skills, Hallock initially worked on solving Vichy French and German Enigma machine cipher systems before being transferred to the Special Problems Section in 1943. Hallock background from Robert L. Benson, Introductory History of VENONA (Fort Meade, MD: Center for Cryptologic History), p. 2; SRH-361, History of the Signal Security Agency, vol. 2, The General Cryptanalytic Problems, pp. 114, 118, 129, 236, 238, 253, RG-457, entry 9002 Special Research Histories, NA, CP. Hallock was also the author of a number of scholarly books, including The Chicago Syllabary and the Louvre Syllabary (Chicago: University of Chicago Press, 1940) and Persepolis Fortification Tablets (Chicago: University of Chicago Press, 1969). For details of Hallock’s breakthrough, see Weekly Report for Section B III b9 for Week Ending 1 October 1943, p. 1; Weekly Report for Section B III b9 for Week Ending 8 October 1945, p. 2; Weekly Report for Section B III b9 for Week Ending 19 November 1945, p. 1, all in RG-457, HCC, box 1114, file SSA BIII Weekly Reports, NA, CP
.
7. David A. Hatch, “Venona: An Overview,” American Intelligence Journal, vol. 17, nos. 1–2 (1996): p. 72.
8. For change in priorities and expansion of SIGINT effort against neutrals and friendly nations, see memorandum, McCormack to Clarke, S.S.B. Priorities, January 26, 1943, RG-457, HCC, box 1432, file: SSS Intercept Priorities, NA, CP; memorandum, Taylor to Clarke and McCormack, S.S.B. Priorities, February 3, 1943, RG-457, HCC, box 1432, file: SSS Intercept Priorities, NA, CP; memorandum, Strong to Chief Signal Officer, March 8, 1943, RG-457, HCC, box 1025, file: C/A Solutions, Intercept Evaluations, 1943–44, NA, CP. For not wanting to be bullied after the end of the war, see Lietuenant ( j.g.) J. V. Connorton, The Status of U.S. Naval Communication Intelligence After World War II, December 17, 1943, p. 9, RG-457, HCC, box 1008, file: Post War Planning Files, NA, CP.
9. For putting distance between the U.S. and British SIGINT efforts, see memorandum, Taylor to Clarke, Cooperation Between United States Signal Intelligence Service and British Y Service, April 5, 1943, RG-457, HCC, box 1417, file: Army and Navy COMINT Regulations and Papers, NA, CP. For secrecy of the SIGINT effort against the USSR, see memorandum, Corderman to Taylor, Draft of “Priorities Schedule,” March 6, 1943, and memorandum, Taylor to Corderman, SPSIS 311.5—General—Draft of “Priorities Schedule,”March 8, 1943, both in RG-457, HCC, box 1432, file: SSS Intercept Priorities, NA, CP.
1: Roller-Coaster Ride
1. Included in the thirty-seven thousand personnel were approximately seventeen thousand assigned to dozens of tactical COMINT collection units stationed overseas. SRH-277, “A Lecture on Communications Intelligence by RADM E.E. Stone, DIRAFSA,” p. 12, RG-457, entry 9002 Special Research Histories, NA, CP. For the number of codes and ciphers being exploited in June 1945, see SSA General Cryptanalytic Branch Orga nization Chart, June 1, 1945, p. B-2, RG-457, HCC, box 1004, file SSA Organization Charts, NA, CP. For 88,747 diplomatic messages, see “The General Cryptanalytic Branch,” in SSA, Annual Report Fiscal Year 1945, General Cryptanalysis Branch (B-3): July 1944–July 1945, RG-457, HCC, box 1380, file General Cryptanalysis Branch Annual Report 1945, NA, CP.
2. Memorandum, Adjutant General to Commanding Generals, Establishment of the Army Security Agency, September 6, 1945; memorandum, Adjutant General to Chief, Military Intelligence Service, Establishment of the Army Security Agency, September 19, 1945; memorandum, Adjutant General to Commanding General, Army Service Forces, Transfer of Signal Security Agency to Army Security Agency, September 21, 1945; memorandum, Assistant Chief of Staff, G-2 to Commanding Generals, Establishment of the Army Security Agency, November 7, 1945; memo for record, General Provisions of the Army Security Agency, May 17, 1946, all in RG-165, entry 421 ABC files, box 269, file: ABC 350.05 (8 Dec 43), sec. 1, NA, CP.
3. SRMN-084, The Evolution of the Navy’s Cryptologic Organization, p. 9, RG-457, NA, CP.
4. Elliott E. Okins, To Spy or Not to Spy (Chula Vista, CA: Pateo Publishing Co., 1985), pp. 150–51; SRH-039, Unit History, 2nd Army Air Force Radio Squadron, Mobile, April 1945–June 1946, p. 9, RG-457, entry 9002 Special Research Histories, NA, CP.
5. Oral history, Interview with Pat M. Holt #1: Years in Journalism, September 9, 1980, p. 13, U.S.
6. SRH-364, History of SSA, p. 237, RG-457, entry 9002 Special Research Histories, NA, CP; National Cryptologic School, On Watch: Profiles from the National Security Agency’s Past 40 Years (Fort Meade, MD: NSA/CSS, 1986), pp. 14–16, NSA FOIA; NSA, oral history, Interview with Frank B. Rowlett, 1976, p. 357, NSA FOIA.
7. The overall strength of the combined army and navy COMINT organizations went from 37,000 on duty on VJ Day to only 7,500 men and women at the end of December 1945. The army COMINT organization’s command strength went from 10,600 men and women on VJ Day plus 17,000 personnel assigned to tactical intercept units to only 5,000 by the end of December 1945. The navy COMINT organization’s staff levels went from 10,051 men and women on duty on VJ Day to only 2,500 personnel on the organization’s rolls at the end of December 1945. For the impact of army personnel losses, see ASA, Summary Annual Report of the Army Security Agency, Fiscal Year 1946, July 31, 1947, p. 7, INSCOM FOIA; memorandum, Johnston to Assistant Chief of Staff, G-2, Report of Signal Security Agency and Second Signal Service Battalion Personnel Strength, December 4, 1945, RG-319, entry 47B Army G-2 Decimal File 1941– 1948, box 568, file 320.2 5/1/45–12/31/45 (31 Dec 44), NA, CP; ASA, “Minutes of 38th Staff Meeting Held 4 December 1945 at 1300,” in SRMA-011, SSS/SSA/ASA Staff Meeting Minutes: 25 November 1942–17 February 1948, pp. 271, RG-457, NA, CP. For the impact of navy personnel losses, see memorandum, Wenger to OP-20, Report of Progress in OP-20-G During Absence of CNC, December 5, 1945, Enclosure 1, p. 1, RG-38, CNSG Library, box 114, file 5750/220 OP-20 Memos Covering Various Subjects 1942–45, part 4 of 5, NA, CP; Op-20-A-vb (5 Jan 1946), Serial: 1002P20, memorandum, Chief of Naval Communications to Chief of Naval Operations, Assistant Chief of Naval Communications for Communications Intelligence—Recommendation for Promotion to the Rank of Commodore, U.S. Navy, January 7, 1946, p. 1, RG-38, CNSG Library, box 81, file 5420/36 Dyer Board 1945, NA, CP.
8. ASA, Summary Annual Report of the Army Security Agency: Fiscal Year 1946, July 31, 1947, pp. 21–22, INSCOM FOIA; SRMA-011, SSS/SSA/ASA Staff Meeting Minutes, p. 265, RG-457, NA, CP; memorandum, OP-23 to OP-02, Future Status of U.S. Naval Communications Intelligence Activities—Comments On, January 16, 1946; memorandum, Redman to OP-02, Future Status of U.S. Naval Communications Intelligence Activities, January 23, 1946; memorandum, Inglis to OP-02, Future Status of U.S. Naval Communications Intelligence Activities, January 25, 1946, all in RG-80, SecNav/CNO Top Secret Decimal File 1944–1947, box 42, file 1946 A8, NA, CP.
9. SSA, “Minutes of 25th Staff Meeting Held 14 August 1945 at 1300,” in SRMA-011, SSS/SSA/ASA Staff Meeting Minutes: 25 November 1942–17 February 1948, p. 216, RG-457, NA, CP; “Minutes of the Fourteenth Meeting of the Army-Navy Cryptanalytic Research and Development Committee,” August 22, 1945, p. 6, RG-38, CNSG Library, box 92, file 5420/169 ANCIB (2 of 2), NA, CP; ASA, Summary Annual Report of the Army Security Agency: Fiscal Year 1946, July 31, 1947, p. 24, INSCOM FOIA; NSA OH-15-82, oral history, Interview with Ann Z. Caracristi, July 16, 1982, p. 29, NSA FOIA.
10. Copies of these decrypts can be found in the collection of T-series messages in RG-457, HCC, box 521, file Decrypted Diplomatic Traffic: T3101–T3200, NA, CP.
11. Andrew and Leslie Cockburn, Dangerous Liaison: The Inside Story of the U.S.-Israeli Covert Relationship (New York: HarperCollins Publishers, 1991), pp. 36–37; NSA-OH-11-82, oral history, Interview with Captain Wesley A. Wright, USN, May 24, 1982, p. 66, NSA FOIA.
12. For ASA military targets, see ASA Descriptive Dictionary of Cryptologic Terms (Laguna Hills, CA: Aegean Park Press, 1997), pp. 4, 11, 21, 23, 36, 62, 65, 95, 111, 113, 143, 150. For OP-20-G’s successes with foreign naval ciphers, see War Diary Report OP-20-G-4A: 1 September to 1 October 1945, October 1, 1945, p. 5, RG-38, CNSG Library, file 5750/160, NA, CP; “Minutes of the Sixteenth Meeting of the Army-Navy Cryptanalytic Research and Development Committee,” October 17, 1945, p. 11, RG-38, CNSG Library, box 92, file 5420/169 ANCIB (2 of 2), NA, CP; G4A War Diary Summary for November 1945, December 4, 1945, p. 1, RG-38, CNSG Library, file 5750/160, NA, CP; G4A War Diary Summary for May 1946, June 6, 1946, p. 1, RG-38, CNSG Library, file 5750/160, NA, CP; memorandum, OP-20-3-GY-A to OP-20-3, Status of Work Report on Spanish, Portuguese, Dutch and French Language Systems, January 16, 1946, RG-38, CNSG Library, box 22, file 3222/85: Non-Japanese Crypto-Systems Processed— Apr 43–Aug 45 (3 of 3), NA, CP.
13. Memorandum, OP-23 to OP-02, Future Status of U.S. Naval Communication Intelligence Activities— Comments on, January 16, 1946, RG-38, CNSG Library, box 114, file 5750/220 OP-20 Memos Covering Various Subjects, 1942–1945 (4 of 5), NA, CP; memorandum, Redman to OP-02, Future Status of U.S. Naval Communication Intelligence Activities, January 23, 1946, and memorandum, Inglis to OP-02, Future Status of U.S. Naval Communications Intelligence Activities,
January 25, 1946, both in RG-80, SecNav/CNO TS, box 42, file 1946 A8, NA, CP.
14. For an example of a Chinese Nationalist military decrypt, see Navy Department, Chief of Naval Operations, Oriental Communication Intelligence Summary, April 11, 1946, RG-38, entry 345 Radio Intelligence Summaries 1941–1946, box 122, file 1-30 April 1946 (2 of 2), NA, CP. For decrypted Chinese Communist radio traffic, see Navy Department, Chief of Naval Operations, Oriental Communication Intelligence Summary, April 26, 1946, RG-38, entry 345 Radio Intelligence Summaries 1941–1946, box 122, file 1-30 April 1946 (2 of 2), NA, CP.
15. Memorandum, Craig to Acting Deputy Chief of Staff, Intelligence on Russia, March 14, 1946, RG-319, entry 154 OPD TS Decimal File 1946–1948, box 75, file P&O 350.05 TS (Section I) Cases 1–44, NA, CP; memorandum, G.A.L. to Hull, Intelligence on Russia, March 22, 1946, RG-319, entry 154 OPD TS Decimal File 1946–1948, box 75, file P&O 350.05 TS (Section I) Cases 1–44, NA, CP; memorandum, Starbird to Hull, Intelligence in Europe, April 3, 1946, RG-319, entry 154 OPD TS Decimal File 1946–1948, box 75, file P&O 350.05 TS (Section I) Cases 1–44, NA, CP.
16. A heavily redacted version of the BRUSA Agreement was recently released to the author, for which see British–U.S. Communication Intelligence Agreement, March 5, 1946, DOCID 3216600, NSA FOIA. See also SRMA-011, SSS/SSA/ASA Staff Meeting Minutes: 25 November 1942–17 February 1948, pp. 293, 321, RG-457, NA, CP; Army-Navy Communication Intelligence Board Organi-zational Bulletin No. 1, June 1945, RG-457, HCC, box 1364, NA, CP; Report to the Secretary of State and the Secretary of Defense (hereafter “Brownell Committee Report”), June 13, 1952, p. 15, NSA FOIA; George F. Howe, “The Early History of NSA,” Cryptologic Spectrum, vol. 4, no. 2, (Spring 1974): p. 13, DOCID: 3217154, NSA FOIA; Thomas L. Burns, The Origins of the National Security Agency: 1940–1952 (Fort Meade, MD: Center for Cryptologic History, 1990), pp. 36–37, 52, NSA FOIA.
The Secret Sentry Page 42