What Just Happened: A Chronicle From the Information Frontier

by James Gleick

  As soon as people conceived of sending messages by telegraph, they worried that their communication was exposed to the world—at the very least, to the telegraph operators, unreliable strangers who could not help but read the words they fed through their devices. Compared to handwritten letters, folded and sealed with wax, the whole affair seemed public and insecure—the messages passing along those mysterious conduits, the electric wires. Vail himself wrote in 1847, “The great advantage which this telegraph possesses in transmitting messages with the rapidity of lightning, annihilating time and space, would perhaps be much lessened in its usefulness, could it not avail itself of the application of a secret alphabet.”♦ There were, he said, “systems”—

  by which a message may pass between two correspondents, through the medium of the telegraph, and yet the contents of that message remain a profound secret to all others, not excepting the operators of the telegraphic stations, through whose hands it must pass.

  This was all very difficult. The telegraph served not just as a device but as a medium—a middle, intermediary state. The message passes through this medium. Distinct from the message, one must also consider the contents of that message. Even when the message must be exposed, the contents could be concealed. Vail explained what he meant by secret alphabet: an alphabet whose characters have been “transposed and interchanged.”

  Then the representative of a, in the permanent alphabet, may be represented by y, or c, or x, in the secret alphabet; and so of every other letter.

  Thus, “The firm of G. Barlow & Co. have failed” becomes “Ejn stwz ys & qhwkyf p iy jhan shtknr.” For less sensitive occasions, Vail proposed using abbreviated versions of common phrases. Instead of “give my love to,” he suggested sending “gmlt.” He offered a few more suggestions:

  mhii My health is improving

  shf Stocks have fallen

  ymir Your message is received

  wmietg When may I expect the goods?

  wyegfef Will you exchange gold for eastern funds?

  All these systems required prearrangement between sender and recipient: the message was to be supplemented, or altered, by preexisting knowledge shared at both ends. A convenient repository for this knowledge was a code book, and when the first Morse line opened for business, one of its key investors and promoters, the Maine congressman Francis O. J. Smith, known as Fog, produced one: The Secret Corresponding Vocabulary;♦ adapted for use to Morse’s Electro-Magnetic Telegraph: and also in conducting written correspondence, transmitted by the mails, or otherwise. It was nothing but a numbered, alphabetical list of 56,000 English words, Aaronic to zygodactylous, plus instructions. “We will suppose the person writing, and the person written to, are each in possession of a copy of this work,” Smith explained. “Instead of sending their communications in words, they send numbers only, or partly in numbers, and partly in words.” For greater security, they might agree in advance to add or subtract a private number of their own choosing, or different numbers for alternate words. “A few such conventional substitutes,” he promised, “will render the whole language a perfectly dead letter to all persons not conusant to the concerted arrangement.”

  Cryptographers had a mysterious history, their secrets handed along in clandestine manuscripts, like the alchemists’. Now code making emerged into the light, exposed in the hardware of commerce, inspiring the popular imagination. In the succeeding decades, many other schemes were contrived and published. They ranged from penny pamphlets to volumes of hundreds of pages of densely packed type. From London came E. Erskine Scott’s Three Letter Code for Condensed Telegraphic and Inscrutably Secret Messages and Correspondence. Scott was an actuary and accountant and, like so many in the code business, a man evidently driven by an obsession with data. The telegraph opened up a world of possibilities for such people—cataloguers and taxonomists, wordsmiths and numerologists, completists of all kinds. Scott’s chapters included not only a vocabulary of common words and two-word combinations, but also geographic names, Christian names, names of all shares quoted on the London Stock Exchange, all the days in the year, all regiments belonging to the British army, registries of shipping, and the names of all the peers of the realm. Organizing and numbering all this data made possible a form of compression, too. Shortening messages meant saving money. Customers found that the mere substitution of numbers for words helped little if at all: it cost just as much to send “3747” as “azotite.” So code books became phrase books. Their object was a sort of packing of messages into capsules, impenetrable to prying eyes and suitable for efficient transmission. And of course, at the recipient’s end, for unpacking.

  An especially successful volume in the 1870s and ’80s was The A B C Universal Commercial Electric Telegraphic Code, devised by William Clauson-Thue.♦ He advertised his code to “financiers, merchants, shipowners, brokers, agents, &c.” His motto: “Simplicity and Economy Palpable, Secrecy Absolute.” Clauson-Thue, another information obsessive, tried to arrange the entire language—or at least the language of commerce—into phrases, and to organize the phrases by keyword. The result is a peculiar lexicographic achievement, a window into a nation’s economic life, and a trove of odd nuance and unwitting lyricism. For the keyword panic (assigned numbers 10054–10065), the inventory includes:

  A great panic prevails in ———

  The panic is settling down

  The panic still continues

  The worst of the panic is over

  The panic may be considered over

  For rain (11310–11330):

  Cannot work on account of rain

  The rain has done much good

  The rain has done a great amount of damage

  The rain is now pouring down in good earnest

  Every prospect of the rain continuing

  Rain much needed

  Rain at times

  Rainfall general

  For wreck (15388–15403):

  Parted from her anchors and became a wreck

  I think it best to sell the wreck as it lies

  Every attention will be made to save wreck

  Must become a total wreck

  Customs authorities have sold the wreck

  Consul has engaged men to salve wreck

  The world being full of things as well as words, he endeavored, too, to assign numbers to as many proper names as he could list: names of railways, banks, mines, commodities, vessels, ports, and stocks (British, colonial, and foreign).

  As the telegraph networks spread under the oceans and across the globe, and international tariffs ran to many dollars per word, the code books thrived. Economy mattered even more than secrecy. The original trans-Atlantic rate was about one hundred dollars for a message—a “cable,” as it was metonymically called—of ten words. For not much less, messages could travel between England and India, by way of Turkey or Persia and Russia. To save on the tariff, clever middlemen devised a practice called “packing.” A packer would collect, say, four messages of five words each and bundle them into a fixed-price telegram of twenty words. The code books got bigger and they got smaller. In 1885 W. H. Beer & Company in Covent Garden published a popular Pocket Telegraphic Code, price one penny, containing “more than 300 one-word telegrams,” neatly organized by subject matter. Essential subjects were Betting (“To what amount shall I back for you at present odds?”), Bootmaker (“These boots don’t fit, send for them directly”), Washerwoman (“Call for the washing to-day”), and Weather—In Connexion with Voyages (“It is far too rough for you to cross to-day”). And a blank page was provided for “Secret Code. (Fill up by arrangement with friends.)” There were specialized codes for railways and yachts and trades from pharmacist to carpetmaker. The grandest and most expensive code books borrowed freely from one another. “It has been brought to the Author’s knowledge that some persons have purchased a single copy of the ‘A B C Telegraphic Code’ for service in compiling Codes of their own,”♦ complained Clauson-Thue. “The Author would intimate that such a
n operation is a breach of the Copyright Act, and liable to become a matter of legal and unpleasant procedure.” This was just bluster. By the turn of the century, the world’s telegraphers, through the medium of International Telegraphic Conferences held in Berne and in London, had systematized codes with words in English, Dutch, French, German, Italian, Latin, Portuguese, and Spanish. The code books prospered and expanded through the first decades of the twentieth century and then vanished into obscurity.

  Those who used the telegraph codes slowly discovered an unanticipated side effect of their efficiency and brevity. They were perilously vulnerable to the smallest errors. Because they lacked the natural redundancy of English prose—even the foreshortened prose of telegraphese—these cleverly encoded messages could be disrupted by a mistake in a single character. By a single dot, for that matter. For example, on June 16, 1887, a Philadelphia wool dealer named Frank Primrose telegraphed his agent in Kansas to say that he had bought—abbreviated in their agreed code as BAY—500,000 pounds of wool. When the message arrived, the key word had become BUY. The agent began buying wool, and before long the error cost Primrose $20,000, according to the lawsuit he filed against the Western Union Telegraph Company. The legal battle dragged on for six years, until finally the Supreme Court upheld the fine print on the back of the telegraph blank, which spelled out a procedure for protecting against errors:

  To guard against mistakes or delays, the sender of a message should order it REPEATED; that is telegraphed back to the originating office for comparison.… Said company shall not be liable for mistakes in … any UNREPEATED message … nor in any case for errors in cipher or obscure messages.♦

  The telegraph company had to tolerate ciphers but did not have to like them. The court found in favor of Primrose in the amount of $1.15, the price of sending the telegram.

  Secret writing was as old as writing. When writing began, it was in itself secret to all but the few. As the mystery dissolved, people found new ways to keep their words privileged and recondite. They rearranged words into anagrams. They reversed their script in the mirror. They invented ciphers.

  In 1641, just as the English Civil War began, an anonymous little book catalogued the many known methods of what it called “cryptographia.” These included special paper and ink:♦ the juice of lemons or onions, raw egg, or “the distilled Juice of Gloworms,” which might or might not be visible in the dark. Alternatively, writing could be obscured by substituting letters for other letters, or inventing new symbols, or writing from right to left, or “transposing each Letter, according to some unusual Order, as, suppose the first Letter should be at the latter End of the Line, the second at the Beginning, or the like.” Or a message could be written across two lines:

  T e o l i r a e l m s f m s e s p l u o w e u t e l

  h s u d e s r a l o t a i h d, u p y s r e m s y i d

  The Souldiers are allmost famished, supply us or wee must yeild.

  Through transposition and substitution of letters, the Romans and the Jews had devised other methods, more intricate and thus more obscure.

  This little book was titled Mercury: or the Secret and Swift Messenger. Shewing, How a Man may with Privacy and Speed communicate his Thoughts to a Friend at any Distance. The author eventually revealed himself as John Wilkins, a vicar and mathematician, later to become master of Trinity College, Cambridge, and a founder of the Royal Society. “He was a very ingenious man and had a very mechanical head,”♦ one contemporary said. “One of much and deep thinking,… lusty, strong grown, well set, broad shouldered.” He was also thorough. If he could not mention every cipher tried since ancient times, he nonetheless included all that could have been known to a scholar in seventeenth-century England. He surveyed secret writing both as a primer and a compendium.

  For Wilkins the issues of cryptography stood near the fundamental problem of communication. He considered writing and secret writing as essentially the same. Leaving secrecy aside, he expressed the problem this way: “How a Man may with the greatest Swiftness and Speed, discover his Intentions to one that is far distant from him.”♦ By swiftness and speed he meant, in 1641, something philosophical; the birth of Isaac Newton was a year away. “There is nothing (we say) so swift as Thought,” he noted. Next to thought, the swiftest action seemed to be that of sight. As a clergyman, he observed that the swiftest motion of all must belong to angels and spirits. If only a man could send an angel on an errand, he could dispatch business at any distance. The rest of us, stuck with Organical Bodies, “cannot communicate their Thoughts so easie and immediate a way.” No wonder, Wilkins wrote, that angels are called messengers.

  As a mathematician, he considered the problem from another side. He set out to determine how a restricted set of symbols—perhaps just two, three, or five—might be made to stand for a whole alphabet. They would have to be used in combination. For example, a set of five symbols—a, b, c, d, e—used in pairs could replace an alphabet of twenty-five letters:

  “According to which,” wrote Wilkins, “these words, I am betrayed, may be thus described: Bd aacb abaedddbaaecaead.” So even a small symbol set could be arranged to express any message at all. However, with a small symbol set, a given message requires a longer string of characters—“more Labour and Time,” he wrote. Wilkins did not explain that 25 = 52, nor that three symbols taken in threes (aaa, aab, aac,…) produce twenty-seven possibilities because 33 = 27. But he clearly understood the underlying mathematics. His last example was a binary code, awkward though this was to express in words:

  Two Letters of the Alphabet being transposed through five Places, will yield thirty two Differences, and so will more than serve for the Four and twenty Letters; unto which they may be thus applied.

  Two symbols. In groups of five. “Yield thirty two Differences.”

  That word, differences, must have struck Wilkins’s readers (few though they were) as an odd choice. But it was deliberate and pregnant with meaning. Wilkins was reaching for a conception of information in its purest, most general form. Writing was only a special case: “For in the general we must note, That whatever is capable of a competent Difference, perceptible to any Sense, may be a sufficient Means whereby to express the Cogitations.”♦ A difference could be “two Bells of different Notes”; or “any Object of Sight, whether Flame, Smoak, &c.”; or trumpets, cannons, or drums. Any difference meant a binary choice. Any binary choice began the expressing of cogitations. Here, in this arcane and anonymous treatise of 1641, the essential idea of information theory poked to the surface of human thought, saw its shadow, and disappeared again for four hundred years.

  The contribution of the dilettantes is what the historian of cryptography David Kahn calls the excited era triggered by the advent of the telegraph.♦ A new public interest in ciphers arose just as the subject bloomed in certain intellectual circles. Ancient methods of secret writing appealed to an odd assortment of people, puzzle makers and game players, mathematically or poetically inclined. They analyzed ancient methods of secret writing and invented new ones. Theorists debated who should prevail, the best code maker or the best code breaker. The great American popularizer of cryptography was Edgar Allan Poe. In his fantastic tales and magazine essays he publicized the ancient art and boasted of his own skill as a practitioner. “We can scarcely imagine a time when there did not exist a necessity, or at least a desire,”♦ he wrote in Graham’s Magazine in 1841, “of transmitting information from one individual to another, in such manner as to elude general comprehension.” For Poe, code making was more than just a historical or technical enthusiasm; it was an obsession. It reflected his sense of how we communicate our selves to the world. Code makers and writers are trafficking in the same goods. “The soul is a cypher, in the sense of a cryptograph; and the shorter a cryptograph is, the more difficulty there is in comprehension,”♦ he wrote. Secrecy was in Poe’s nature; he preferred mystery to transparency.

  “Secret intercommunication must have existed almost contemporaneously wi
th the invention of letters,” he declared. This was for Poe a bridge between science and the occult, between the rational mind and the savant.♦ To analyze cryptography—“a serious thing, as the means of imparting information”—required a special form of mental power, a penetrating mind, and might well be taught in academies. He said again and again that “a peculiar mental action is called into play.” He published as challenges to his readers a series of substitution ciphers.

  Along with Poe, Jules Verne and Honoré de Balzac also introduced ciphers into their fiction. In 1868, Lewis Carroll had a card printed on two sides with what he called “The Telegraph-Cipher,” which employed a “key-alphabet” and a “message-alphabet,”♦ to be transposed according to a secret word agreed on by the correspondents and carried in their memories. But the most advanced cryptanalyst in Victorian England was Charles Babbage. The process of substituting symbols, crossing levels of meaning, lay near the heart of so many issues. And he enjoyed the challenge. “One of the most singular characteristics of the art of deciphering,” he asserted, “is the strong conviction possessed by every person, even moderately acquainted with it, that he is able to construct a cipher which nobody else can decipher. I have also observed that the cleverer the person, the more intimate is his conviction.”♦ He believed that himself, at first, but later switched to the side of the code breakers. He planned an authoritative work to be known as The Philosophy of Decyphering but never managed to complete it. He did solve, among others, a polyalphabetic cipher known as the Vigenère, le chiffre indéchiffrable, thought to be the most secure in Europe.♦ As in his other work, he applied algebraic methods, expressing cryptanalysis in the form of equations. Even so, he remained a dilettante and knew it.