Intelligence in War: The Value--And Limitations--Of What the Military Can Learn About the Enemy

by John Keegan

  They anticipated cracking the system completely in 1941. By then, however, all spare cryptanalytic manpower in Washington had been diverted to a new task: decrypting the Japanese diplomatic traffic enciphered on a new ciphering machine, known to the Americans as Purple. The Purple machine (Type 97 to the Japanese) was designed to achieve the same effect as Enigma—the automatic production of an almost infinitely variable cipher—but differed from it in construction. It was less mechanical, having no rotors, but instead a set of telephonic switches, connected to two typewriters. The first was used to input the text, the second to print out the encipherment for transmission. In between, the switches moved the incoming electrical current to achieve alphabetic substitutions. Because Japanese is a syllabic, not alphabetic, language, however, all texts had first to be written in an alphabetic equivalent; and for an inexplicable reason, equivalent to the Germans’ double-encipherment of the operator’s chosen indicator at the beginning of a transmission, the Purple machine’s switches enciphered vowels and consonants separately; the number of vowel substitutions was considerably smaller than that of consonants, and once that was recognised, a way into Purple was found.8

  The breaking of Purple—its product was known as Magic, the equivalent of British Ultra—would eventually yield huge intelligence advantages to the Americans, principally through the decipherment of the messages sent by the Japanese ambassador in Berlin, Baron Oshima, to Tokyo throughout the war, which revealed intimate details of Hitler’s capabilities and intentions. At the critical moment before Japan’s surprise attack on America in December 1941, however, Purple revealed little, while JN-25, the relevant Japanese naval code, was not useful for two reasons: the first was that the Imperial Navy attempted to observe, as far as was possible, radio silence in the period preparatory to the descent on Pearl Harbor; the second was that OP-20-G lacked the staff necessary to deal with the volume of intercepted traffic. The U.S. Navy’s Historical Center has now compiled a list of significant messages intercepted—but not decrypted—in the weeks before Pearl Harbor that bear on the issue of foreknowledge. Some, if read in real time, must have alerted a wary admiral to the danger threatening his fleet; in practice, the messages were bundled up and not decrypted and translated until September 1945, a month after the Japanese war had ended.9

  Pearl Harbor devastated not only the U.S. Navy’s Pacific Fleet but also OP-20-G’s Pacific outstations. While its Hawaiian station (HYPO) continued to operate, its outpost in the Philippines (CAST) first withdrew into a tunnel in Corregidor, then was evacuated to Australia. American naval intelligence in the Pacific was thus reduced to HYPO, the joint station in Australia and a branch in the British Combined Bureau in Ceylon. The British had had some inter-war success in attacking Japanese naval codes but, in the climate then current, with both the American services and the public adamant about revenge against the Japanese, the prime responsibility for breaking back into Japanese naval traffic lay with OP-20-G. The Commander-in-Chief Pacific, Admiral Chester Nimitz, was on tenterhooks against a renewed Japanese naval offensive; he equally hoped to profit, perhaps from a Japanese mistake but preferably an intelligence coup, and inflict a defeat on the enemy.

  THE COURSE OF JAPANESE CONQUEST

  There was little sign of stemming the tide of Japanese conquest in the last weeks of December 1941 and the first three months of 1942. Its progress seemed inexorable. Attacks launched out of Thailand quickly collapsed the British defences of northern Malaya. The new battleship Prince of Wales and the battlecruiser Repulse, operating without air cover in an effort to intercept Japanese coastwise landings, were sunk by bombers launched from French Indo-China on 10 December. Singapore, the great trading city at the tip of the peninsula, was ignominiously surrendered to an inferior Japanese force on 15 February. The British island of Hong Kong and the American islands of Wake and Guam, all indefensible, were captured on 25 December, 23 December and 10 December; the garrisons of Hong Kong and Wake put up a heroic, hopeless resistance. Burma was invaded in January and conquered by May. The invasion of the Dutch East Indies began in January also and was completed by March; there were several attempts by a combined Australian-British-Dutch-American fleet (ABDA) to check the Japanese amphibious campaign, culminating in the Battle of the Java Sea, on 27 February; a miscellaneous collection of Allied ships, bravely commanded by the Dutch Admiral Karel Doorman, but outnumbered and unable to intercommunicate, was overwhelmed. Meanwhile, General Douglas MacArthur was conducting the defence of the Philippines, the attack on which began on 8 December with a devastatingly successful Japanese air raid. The American and Filipino defenders were forced to withdraw as soon as the Japanese landings began, but in January succeeded in establishing a fortified line across the Bataan peninsula. There for the next three months they sustained a heroic defence, inflicting on the Japanese the only setbacks suffered on land during their great campaign of conquest. Eventually, however, shortage of food and supplies forced a surrender in April, and in May of the offshore island of Corregidor.

  The defeat in the Java Sea had destroyed the American Asiatic Fleet, originally based in the Philippines, leaving only its submarines to survive. The Pacific Fleet, based at Pearl Harbor, remained operational; but the loss and disablement of its eight battleships had transformed its composition. From being a big-gun capital force, it had become necessarily an aircraft carrier fleet. The three carriers based on Pearl—Enterprise, Lexington and Saratoga—had been absent on 7 December. The navy’s three others, Wasp, Hornet and Ranger—were elsewhere. It was around these six units that the Pacific Fleet was to concentrate the rest of its strength and design a new offensive strategy that, in a succession of brilliant victories, would halt and then reverse Japan’s Pacific onslaught.

  In the spring of 1942, the hopes of Japan’s strategic optimists seemed to have been wholly realised and the fears of her strategic pessimists altogether disproved. Yamamoto, the only Japanese admiral who knew America well, had predicted that he could “run wild for a year or six months,” but thereafter he foresaw only the gathering strength of American industry. The magnitude of Japan’s victory seemed to make economic imbalances irrelevant. The Americans, together with their European allies, had been beaten; it was henceforth a question only of where Japan should strike next to capitalise on its success.

  There were two schools of thought among Japanese planners, the “Southern” school and the “Central Pacific” school; the Central Pacific school was entirely a naval one, the Southern school also involved the army. The Central Pacific school held that the carrier striking force should resume the attack on Hawaii, to dispose for good of the U.S. fleet’s ability to intervene in Pacific grand strategy. The Southern school took a more indirect view, identifying Australia as a potential base for an Anglo-American counteroffensive but also wanting to eliminate British maritime power in the Indian Ocean, thereby weakening Britain’s, as well as China’s, ability to wage war in Burma and thus laying the basis for a Japanese offensive against Britain’s Indian empire itself.

  In the early course of the debate, the navy appeared to accept the objections raised against its Central Pacific concept and, during March, deployed two carrier striking forces to the Indian Ocean. One, commanded by Admiral Nagumo, who had led at Pearl Harbor, struck the British base in Ceylon, sank a British carrier, Hermes, two large cruisers, Dorsetshire and Cornwall, and forced the squadron of old R-class battleships to withdraw to east Africa. Meanwhile, the smaller Japanese carrier task force under Admiral Ozawa roamed the Bay of Bengal, sinking 100,000 tons of commercial shipping in five days. Those with a long memory would remember the cruise of the Emden, though Ozawa’s depredations were far more brutal.

  The Americans, meanwhile, were not wholly dormant. On 20 February a task force organised around Lexington had attacked Rabaul, the old German base in the Bismarcks, and inflicted heavy losses on the bomber force sent to drive it off; the Japanese lost eighteen aircraft, the Americans two. Then in April an even more daring raid was
mounted. President Roosevelt had been pressing for an American attack on the Japanese home islands for some time. The mission seemed infeasible, since it was too dangerous to risk the Pacific Fleet’s few aircraft carriers in Japanese home waters, while America’s remaining island airfields were too distant to serve as bases for land bombers. In mid-January, however, Captain Francis Low, operations officer to Admiral Ernest King, Chief of Naval Operations, proposed embarking land bombers, which had a range greatly exceeding that of any maritime aircraft, on an aircraft carrier and sailing it to within striking distance of Tokyo.

  The idea seemed fantastic, but Colonel James Doolittle, one of the Army Air Force’s inter-war bomber pioneers, who was put in charge of planning the mission, determined to dissolve the difficulties. He selected the B-25 medium bomber as the best aircraft available and set sixteen crews training in Florida to learn the technique of very short take-offs. After a month of preparation, the crews watched their B-25s lifted by crane aboard the new aircraft carrier Hornet at Alameda Naval Air Station, California, then embarked for the unknown. They had not been told where they were bound. On 13 April, Hornet and its escorts met Enterprise off Midway, the last surviving American island outpost in the North Pacific, and set course for Japan. The plan, of which the crews had only just been told, was to fly the bombers off once they were within 500 miles of the Japanese capital, to release their bombs under cover of darkness and then fly on to crash-land in areas of China not occupied by the Japanese.

  As they approached take-off point, the Americans discovered that the plan had miscarried. Yamamoto, anticipating just such a revenge raid, had established a line of picket boats 600 to 700 miles eastward of the home islands. American radar, and then visual reconnaissance, identified first one, then a second, then a third picket boat. Admiral William Halsey, in command of the joint task force, recognised that further changes of course would not evade interception. It was decided to fly off Doolittle’s force at once, even though they would therefore have to go 650 instead of 500 miles to reach their targets and bomb by day instead of night. In heavy weather, with waves breaking over the bow, all sixteen of Doolittle’s bombers took off successfully, reached Tokyo, dropped their bombs and flew on to China; some crews crash-landed, others bailed out; of the eighty who flew, seventy-one survived.10

  The material effect of the Doolittle raid was insignificant; few citizens of Tokyo were aware that they had been bombed. The psychological effect on the Japanese high command was decisive. Committed to the protection of the body of the Emperor as the supreme value of the warrior creed, Japan’s admirals felt dishonoured by the attack. They had failed in an overriding duty. Plans for invading Australia were immediately postponed and all thoughts refocused on the Central Pacific, with the idea of terminating for good the U.S. Pacific Fleet’s ability to strike at the home islands. Hawaii was still too distant from the Imperial Navy’s centres of power, and too well defended, to be attacked at once. It was seen, however, that its outlier, the tiny island of Midway, could be used as a point of attraction, onto which the surviving American carriers could be drawn by the threat of invasion and then destroyed by the concentration of overwhelming force.

  Japan’s strategic position in April 1942 was extremely advantageous. Her main aim in resort to war had been to take possession of a perimeter, delineated by the Central and South Pacific island chains, which would eliminate American, British and Dutch naval power in the area of command, isolate China from support, and dominate the long sea route from California to Australia, which the Japanese identified, correctly, as the base the Americans would use to mount a counteroffensive. Much of the perimeter, thanks to the mandating of the ex-German islands to Japan, lay in her possession before the war began. The rest was supplied by the capture of Wake, Guam and the Dutch East Indies; the important land masses inside the perimeter, the Philippines, Malaya and Burma, were acquired by subsidiary offensives.

  Despite the overwhelming success of Japan’s initial campaign of conquest, however, gaps remained in its strategic perimeter even in April 1942. Only the northern half of New Guinea had been conquered, and the Solomon Islands, beyond the Bismarck chain, remained in dispute. The Americans still potentially enjoyed a way round the tail of the New Guinea “bird” to Australian ports. Even while preparing the Midway operation, therefore, the Japanese decided to complete their conquest of New Guinea by sending a carrier force into the Coral Sea, between the great island and the north coast of Australia, to capture Port Moresby on the south coast and so facilitate the advance of the Japanese ground troops across the island’s central spine, the Owen Stanley range.

  Japan had already operated in the Coral Sea, bombing Darwin, in Australia’s Northern Territories, a week before the Battle of the Java Sea in February 1942. Its plan now was to send a tripartite naval mission, one element to land a force at Port Moresby, a second to capture Tulagi in the Solomons and a third striking force, of two large carriers, Shokaku and Zuikaku, to cover both operations. American cryptographers had identified Japanese intentions, direction-finders had located the position of the main Japanese units and Admiral Nimitz detached two of his precious carriers, Lexington and Yorktown, to deal with the intruders.

  A very confused encounter, known as the Battle of the Coral Sea, ensued. Japanese aircraft found an American oiler and a destroyer, the Neosho and the Sims, mistook the first for a carrier, the second for a cruiser, sank Sims, damaged Neosho and returned to their mother ships exultant. American aircraft meanwhile found the force covering the Port Moresby landing ships, sank the small Japanese carrier Shoho and returned equally exultant. On the following day, the main forces found each other, Lexington was sunk, Yorktown and Shokaku were both damaged but withdrew to repair, Zuikaku was not touched. The Americans reckoned Coral Sea a victory, since it prevented the capture of Port Moresby. The Japanese, in terms of ship losses, had reason to regard it as a victory themselves.

  The Coral Sea left the balance of carriers in the two Pacific navies as follows: Japan, Zuikaku, Shokaku, Hiryu, Soryu, Kaga, Akagi and the small carriers Ryujo and Zuiho; the United States, Saratoga, Wasp, Ranger, Enterprise, Yorktown and Hornet. Actual numbers were smaller than paper strength. Of the Japanese carriers, Shokaku had retired to Truk, in the Carolines, to repair; Zuikaku had lost so many of its aircraft at the Coral Sea that it had been withdrawn to refit; Ryujo and Zuiho were judged too small to take part in a major fleet action. On the American side, Wasp and Ranger were absent in the Mediterranean, having been generously lent by President Roosevelt to ferry fighters to the besieged island of Malta, while Yorktown was in dock in Pearl Harbor. Hit by an 800-pound bomb at the Coral Sea that had penetrated to her fourth deck, killed sixty of her crew and started a serious fire, she was also leaking from multiple splinter holes. The dockyard estimated that she needed a ninety-day repair. Admiral Nimitz announced that he needed her operational in three days. She entered dry dock on 27 May and, after 1,400 men had worked round the clock for two days, was floated out on the morning of the 29th. During the afternoon she embarked new aircraft to replace those lost at the Coral Sea and at 9 a.m. on 30 May she left to join the fleet. Nimitz had organised his carriers into the two task forces: TF 16, under Admiral Raymond Spruance, temporarily replacing Halsey, who was in hospital ashore, consisting of Hornet and Enterprise; Yorktown, with its escorts, was to form TF 17, under Admiral Jack Fletcher. Their role was to find and destroy the Japanese main fleet.

  American cryptanalysis had alerted the high command of the Pacific Fleet to a forthcoming Japanese operation. After the raid into the Indian Ocean against the British, and the frustrated offensive of the Coral Sea, it was obvious that their carriers would strike again. The question was, where? The Americans had no inkling of the effect the Doolittle raid had had on the Japanese sense of honour. There was evidence, however, from intercepted Japanese signals, that the next offensive would be in the Central Pacific. It was in that direction that TF 16 and TF 17 were headed.

  MAGIC AND
MIDWAY

  The first indication that the Japanese Combined Fleet might return to the Central Pacific, after its support of operations around the Philippines, Malaya and the Dutch East Indies in December through February, came on 5 March, not through signals intelligence but because of a minor Japanese bombing attack on Hawaii. The Americans correctly concluded that the attack had been launched from the Marshall Islands, via a refuelling stop at an isolated oceanic anchorage known as French Frigate Shoals. What was important for the future about the 5 March raid was that the American intercept stations were able to identify it with something denoted in Japanese codes as the K operation. The significance of K became more apparent on 6 May, when HYPO, the Hawaiian station of American cryptanalytic intelligence, deduced that K was part of an encrypted geographical designator standing for Hawaii. The American codebreakers were beginning to recognise that the Japanese used three (later two) letter groups to denote geographical objectives, those in the American zone of operations beginning with A (hence AK for Hawaii), those in the British zone beginning with D and those in the Australian zone with R.11

  This was an important breakthrough but its value was set back by the introduction of new security measures in the Japanese fleet, consisting mainly of a change of call signs between ships but also between shore and ship. The changes greatly complicated the codebreakers’ ability to identify the location of individual ships and the composition of fleets.12 Their difficulties were compounded by the retreat of the main Japanese naval units into radio silence, as before Pearl Harbor. On 13 March the American cryptanalysts broke into the main naval code, JN-25, but that success was shortly negated by the Japanese adoption of a new system of cipher additives to the code groups. Just before the change the Americans got a crucial insight into Japanese intentions via a request from an unidentified ship for a supply of charts, clearly indicating the interest of the Japanese fleet in the Hawaiian group of islands and its western outliers, which included Midway. Nimitz, commanding the U.S. Pacific Fleet, accordingly concluded that he was faced by four possible Japanese operations: an attack on Midway-Hawaii; an attack on the Aleutian Islands, outliers of American Alaska; an attack on other Central Pacific islands; a renewed attack on the islands of New Guinea, any of those operations to be launched between 25 May and 15 June.13