The Codebreakers — Volume 2 — Riverbank — William and Elizebeth Friedman and the Index of Coincidence

2.1 About This Volume

Every thread in this series begins in the same unlikely place: a private country estate at Geneva, Illinois, about forty miles west of Chicago, owned by a cotton merchant with an honorary colonel’s title and a taste for science, fringe scholarship and eccentric living. Between 1916 and 1920 it was, in effect, the national cryptologic laboratory of the United States. It was where William Friedman learned the subject, where he and Elizebeth Smith learned it together, where the first Americans were formally taught it, and where Friedman wrote the pamphlets — above all The Index of Coincidence and Its Applications in Cryptography — that turned it into a mathematical discipline.

This volume tells how that happened, what the two Friedmans learned at Riverbank and how they taught it, and then follows the two careers that grew out of it. His took him into the War Department and, eventually, to PURPLE and SIGABA (Volumes 3, 5 and 9). Hers took her into the Navy, the Coast Guard and the Treasury, where for more than a decade she was the most effective codebreaker the U.S. government had against organised crime, and then into the Second World War against German spy networks. The statistics are left to Volume 6; here the subject is where the ideas came from, how they were put to work and how they were passed on.

2.2 Colonel Fabyan’s Laboratory

George Fabyan had made his fortune in the family textile business. His title was honorary — conferred by the governor of Illinois for his part in the commission that negotiated the end of the Russo-Japanese War in 1905 — but he liked it, and Riverbank, his estate on the Fox River, reflected a man who could afford to follow his curiosity. It had laboratories for acoustics, chemistry and genetics, a Dutch windmill, a Japanese garden, greenhouses, a menagerie, and a house in which some of the furniture hung from the ceiling on chains. Its staff of hired scientists, Elizebeth remembered, lived the life of the “minor idle rich” in the evenings (Chiles, in The Friedman Legacy, p. 197; Library copy). Its acoustics laboratory still operates under the Riverbank name.

Fabyan’s most expensive enthusiasm was the theory that Francis Bacon had written the works of Shakespeare and hidden the fact, in cipher, in the printed texts. Bacon had indeed invented a cipher — a “biliteral” alphabet in which each letter is represented by five symbols of two kinds, and which can be concealed by printing an innocent text in two slightly different typefaces. A Boston scholar named Elizabeth Wells Gallup believed she could see the two founts in the original Shakespeare folios and read Bacon’s hidden messages in them. Fabyan installed her at Riverbank with a staff of assistants.

2.3 Two Recruits

2.3.1 Elizebeth Smith

Elizebeth Smith was born in Huntington, Indiana, in 1892, the youngest of nine children of a Quaker dairyman and banker; the unusual spelling of her name was her mother’s, who would not have her called “Eliza”. She studied briefly at Wooster College and graduated from Hillsdale College in Michigan in English literature, with Latin, Greek and German along the way, and worked for a year as a high-school principal. In 1916 she went to the Newberry Library in Chicago, drawn — the NSA’s biography supposes — by its Shakespeare folio. A librarian who heard of her enthusiasm telephoned Fabyan, Fabyan came to meet her, and she was hired on the spot to help Mrs. Gallup (NSA Hall of Honor, Elizebeth S. Friedman).

2.3.2 William Friedman

Wolfe Frederick Friedman was born in Kishinev, in imperial Russia, on 24 September 1891, and brought to Pittsburgh as a baby; his name became William when his father was naturalised. He took a B.S. in genetics at Cornell in 1914, did graduate work there, and in 1915 was hired by Fabyan to run the Riverbank genetics department (The Friedman Legacy, biographical sketch; Library copy). He was drawn into the Baconians’ work — and, the NSA’s biography observes drily, into cryptology through “his concurrent interest in Elizebeth Smith” (NSA Hall of Honor, William F. Friedman). They married in May 1917.

What the Bacon project gave them, though neither believed its conclusions for long, was an education. To test Gallup’s readings they had to study every cipher system they could find, read the whole published literature of the subject, and learn to ask whether a claimed decipherment was real or the product of wishful pattern-seeking. It was a training in suspicion that served them all their lives. Forty years on, the two of them published The Shakespearean Ciphers Examined (1957), which disposed of the Baconian case with such technical authority that it won awards from the Folger Shakespeare Library and the American Shakespeare Festival Theatre. On one page they buried their own biliteral cipher in an italicised phrase, set in two founts. It reads: “I did not write the plays. F. Bacon.”

2.4 The Department of Ciphers

In the latter part of 1916 Fabyan set up a Department of Ciphers at Riverbank. Callimahos, who heard the story from Friedman, recorded that it was “first headed by Miss Elizebeth Smith and later by Mr. Friedman, who took over both the Department and Miss Smith” (Callimahos, in The Friedman Legacy, p. 192, note 4; Library copy). Because none of the State, War, Navy or Justice Departments had any cryptanalytic unit of its own, Fabyan offered Riverbank’s services, and the government accepted.

Friedman described the arrangement in one of his 1963 lectures to NSA staff:

There had been, it is true, since the autumn of 1916, a very small group of self-trained cryptanalysts, sponsored and supported by a private citizen named Colonel George Fabyan, who operated the Riverbank Laboratories at Geneva, Illinois. I served as leader of the group, in addition to other duties as a geneticist of the Laboratories. Riverbank, through George Fabyan, had initiated and established an unofficial, or at most, a quasi-official relationship with the authorities in Washington, so that it received from time to time copies of cryptographic messages obtained by various and entirely surreptitious means from telegraph and cable offices in Washington and elsewhere in the United States.

— W. F. Friedman, Lecture V (1963), in The Friedman Legacy, p. 106

Most of what came was Mexican government traffic, in Vigenère-type ciphers “using mixed sequences with relatively long key phrases”, and Riverbank solved “all or nearly all” of it, usually returning the solutions promptly. The work taught the two of them something no book did: that real traffic, sent by tired clerks under pressure, is full of repetitions, stereotyped openings and closings, and errors, and that these are the cryptanalyst’s best friends.

2.4.1 Learning from a manual

Friedman was frank about how little the self-taught Riverbank group had to learn from. In English there was almost nothing but a small book by Captain Parker Hitt, Manual for the Solution of Military Ciphers (Fort Leavenworth, 1916), which Fabyan managed to obtain, and a pamphlet by Lieutenant Joseph O. Mauborgne, An Advanced Problem in Cryptography and its Solution (1914), on the Playfair cipher. “It was the succinctness of Parker Hitt’s manual that caused us much work and perspiration in our self-training at Riverbank,” Friedman recalled, “but we later came to know and admire its author” (Lecture V, p. 107). Mauborgne would become Chief Signal Officer of the Army in 1937, and the commander who received the PURPLE break in 1940.

2.4.2 “This cipher is absolutely undecipherable”

A story Friedman liked to tell shows both his luck and Elizebeth’s part in it. In 1917 the British proposed to use, for joint communications with the United States, a geared disk cipher device originally designed by Sir Charles Wheatstone, regarded as unbreakable if the two mixed alphabets were unknown. Someone in Washington suggested having Riverbank test it. Five short test messages were sent. Friedman recovered the cipher alphabet — a sequence built from the keyword CIPHER — and then stalled on the plain alphabet. He asked Elizebeth for the first word that came into her head when he said “cipher”; she said “machine”; the plain sequence was built on MACHINE. One of the messages read: “This cipher is absolutely undecipherable” (Callimahos, in The Friedman Legacy, pp. 189–190). Accounts differ on the details — a later retelling has the Friedmans breaking all five messages in three hours — but the moral the school drew from it did not change: no system should be adopted until people who know how to break ciphers have attacked it.

2.5 The First Classes

When the United States entered the war in April 1917 it had, as Friedman put it, “no organizations or technical groups whatever, either for intercepting enemy communications or for studying them, let alone solving such communications”. Fabyan offered to train officers. “Soon after the United States declared war on Germany, Colonel Fabyan established a school for training at Riverbank, and he invited the services to send him army and navy officers to learn something about cryptology in formal courses established for the purpose. Each course lasted about six weeks, full time” (Lecture V, p. 106). The courses were run by the Friedmans and paid for entirely by Fabyan — “many thousands of dollars, never repaid by the government”, Friedman noted pointedly.

To teach the classes, the Friedmans had to write the material. The problems and exercises they prepared survive in the NSA archives, which Friedman thought “still of much interest as curiosities of U.S. cryptologic history”. So do the pamphlets that grew out of them.

After the last class Friedman was commissioned a first lieutenant and sent to France, where he served in the Radio Intelligence Section of the American Expeditionary Forces’ General Headquarters at Chaumont — designated G-2, A-6 — working on German Army field codes. There he learned from real wartime traffic how armies actually used codes, and what went wrong: that German messages began and ended with ritual phrases, and that when a new code or cipher was introduced, enemy clerks were most likely to make a serious error that exposed it (Chiles, p. 198). He also saw what could be learned from traffic itself — direction-finding bearings, the volume and flow of messages — without reading a word, and later showed his NSA audiences what he believed was one of the first charts in history of intelligence derived from what is now called traffic analysis (Lecture V, p. 108).

2.6 Eight Pamphlets

Between 1917 and 1922 Fabyan printed, at his own expense, a numbered series of Riverbank Publications. Eight of them, Nos. 15 to 22, were on cryptology; all were by Friedman, two with co-authors. Together they were the first serious body of cryptanalytic literature written in the United States.

The titles show the shape of the subject as Friedman saw it. Several are about reconstructing alphabets — recovering the mixed sequence a cipher clerk had used from the traces it left in ciphertext, the skill that would recover PURPLE’s sixes in 1939. Several are about particular classes of system: running-key ciphers, geometrical transpositions, “several machine ciphers”. One is a set of synoptic tables, a guide to diagnosing which kind of system an unknown cryptogram belongs to. The approach throughout is that of a scientist classifying species: identify the type, then apply the method for that type.

No. 16, Methods for the Solution of Running-Key Ciphers (1918), dealt with polyalphabetic ciphers whose key is a long passage of text, such as a page of a book, and showed how messages enciphered with the same running key could be solved together. No. 20, Several Machine Ciphers and Methods for Their Solution, attacked mechanical devices. Both carried the lesson that would run through Friedman’s career: designers overrate the security of their systems because they think about the key space and not about the analyst.

2.6.1 The Index of Coincidence

The last of the series was different in kind. The Index of Coincidence and Its Applications in Cryptography (Riverbank Publication No. 22) was written in 1920, after the Friedmans returned from Washington to Riverbank, and printed in 1922 with a Paris imprint; a French edition, L’indice de coïncidence et ses applications en cryptographie, also circulated in Europe, and the young Lambros Callimahos read it in Paris in 1934 (Callimahos, in The Friedman Legacy, p. 185; Library copy). The Signal Corps reissued a revised edition in 1935 for its own students.

The idea at its heart is simple enough to put in a sentence. Lay two texts one above the other and count the columns in which the same letter appears in both. If both are ordinary English, such coincidences are relatively common, because E, T, A and a few other letters are common; if both are random letters, coincidences fall to one column in twenty-six. Friedman found that the rate for English plain text was about 6.6 or 6.7 per hundred — “about seven percent of the time”, as one later popular account put it — and that each language had its own characteristic rate (Chiles, p. 198).

The power of the idea lies in what survives encipherment. A polyalphabetic cipher scrambles the letters of each message, but if two messages were enciphered with the same key starting at the same point, the same plaintext letter in the same column becomes the same cipher letter in both — so the plaintext coincidence rate shows through in the ciphertext. A single number, computable by a clerk with a tally sheet, therefore tells the analyst whether two messages are “in depth”, whether a long message uses one alphabet or many, and what the period of a repeating key is likely to be. It turned a matter of hunches into a matter of measurement.

Contemporary and later judgements were unanimous. David Kahn wrote that the pamphlet “must be regarded as the most important single publication in cryptology. It took the science into a new world.” Callimahos, never one to flatter Friedman’s mathematics — he liked to tell students that Friedman had only freshman college mathematics and had once got a probability badly wrong in print — called the 1920 paper “the pioneer paper in cryptomathematics” (The Friedman Legacy, pp. 188, 198). Its mathematics is not deep; its significance is that it was the first time anyone had treated cryptanalysis as a problem in statistical inference — reasoning from counts to hypotheses with a measurable chance of being wrong. Volume 6 works the method in full; Volume 12 follows it into Solomon Kullback’s information theory.

2.7 To Washington

Early in 1921 the Army offered both Friedmans six-month contracts as civilian “code experts” in Washington. They went, warily — “wondering,” one account puts it, “whether [Fabyan’s] tenacious grasp would reach to Washington” — and never returned. William was hired permanently by the Signal Corps in 1922 as its chief cryptographer, and for the next eight years he was, in effect, the Army’s entire cryptologic staff in Washington: he compiled its codes, tested its ciphers and wrote its manuals. In 1923 he published Elements of Cryptanalysis, the first of his Army texts.

2.7.1 The Hebern machine

The first great test of the Riverbank method came from the Navy. Edward Hebern, a Californian inventor, had built an electrical cipher machine in which the current passed through rotating wired disks — rotors — the principle of the Enigma and of most cipher machines of the next forty years. The Navy was interested and, on Friedman’s advice, the Signal Corps bought two of Hebern’s five-rotor models late in 1923. In the spring of 1924 the Navy asked Friedman to test the machine and gave him ten test cryptograms prepared by its Code and Signal Section. He solved them, and developed techniques by which, under certain conditions, solution could be achieved “even without possession of the code wheels” (Safford, history of the Mark II ECM, NSA REF ID A60492, p. 11; Library copy).

Elizebeth remembered what it cost him. “He was discouraged to the point of blackout. Probably he sat for six weeks before he thought of a way to attack it… It was all resolved in the end by the Index of Coincidence” (Chiles, p. 198). Callimahos called it “the first solution in history of a wired-rotor cipher machine”, achieved with only ten messages and a theory of coincidence that Friedman worked out for the purpose. Friedman wrote the solution up in a secret technical paper completed in 1924 but not printed until 1934, Analysis of a Mechanico-Electrical Cryptograph, Part I; a second test, on Hebern’s improved 1930 model, followed in 1932 and became Part II. Those papers were used, Safford recorded, “only in the SIS for the advanced training of a very limited number of students” — and their printing at the Government Printing Office, without consulting the Navy, caused a quarrel between the services that is part of Volume 9’s story.

The Hebern solution taught Friedman the lesson that defined his later career as a designer: a rotor machine whose wheels step regularly, like an odometer, is vulnerable no matter how many wheels it has, because the regularity lets the analyst treat the machine’s parts one at a time. He spent the next ten years looking for a remedy.

2.8 Elizebeth’s War on the Rum-Runners

Elizebeth Friedman’s own career, in the NSA’s phrase, “embraces cryptology against international smuggling and drug running in various parts of the world”. She worked as an assistant cryptanalyst for the War Department in 1921–22 and for the Navy in 1923. Then Prohibition gave her a vocation.

By the mid-1920s the liquor smuggling trade had become an industry, organised by syndicates with radio networks to direct their “mother ships” off the coast and the fast boats that ran cargo ashore. Their messages were encoded and enciphered. In 1927 Elizebeth went to work in the Coast Guard’s offices in Washington as a “special agent”, on loan from the Department of Justice, and attacked the rum-runners’ traffic. The early systems were simple; as profits rose and larger syndicates took over, they became more elaborate. In her first three years, she and her assistants solved some 12,000 messages in dozens of different systems. In 1928 she moved to the Customs service of the Treasury, and in 1931 Congress approved her proposal for a seven-person cryptanalytic section at headquarters (The Friedman Legacy, pp. 199–205; Library copy).

She ran that section the way her husband ran his — as a school. “As her cryptanalytic responsibilities began to mount, Mrs. Friedman sensed the need to teach other analysts cryptanalytic fundamentals,” the NSA’s biography records; teaching them freed her for “the more atypical new systems as they cropped up” and let her stay “one step ahead of the smugglers”.

2.8.1 In court

Her work had a feature that her husband’s lacked: it had to be explained to juries. Her decipherments were evidence, and she was the witness who had to defend them. Between 1928 and 1930 she testified in smuggling cases in Galveston, Houston and New Orleans. In 1933 she was the star prosecution witness in the federal court at New Orleans against the Consolidated Exporters Company, which the NSA describes as “Prohibition’s largest and most powerful bootlegging ring”; her solutions linked its ringleaders to the operations of the vessels, and thirty-five men were indicted for conspiracy to violate the National Prohibition Act. In 1937 the Canadian government brought her to Vancouver to testify against an opium ring whose messages she had read in a code involving Chinese without knowing Chinese; a Pacific coast columnist observed that the convicted men would have seven years “to devise a code that a woman couldn’t break” (The Friedman Legacy, p. 205).

The most diplomatically important of her cases involved the I’m Alone, a Canadian-registered schooner built for the liquor trade, which a Coast Guard cutter chased into international waters off Louisiana in March 1929 and sank with the Canadian flag flying. Canada claimed $365,000 for ship and cargo; the U.S. contended the ship was really American-owned. Elizebeth’s decryption of twenty-three coded telegrams, subpoenaed from cable-company files, between Belize and an unregistered code address in New York identified the owners and established American ownership, removing the basis for most of the claim (The Friedman Legacy, pp. 205–206). While the case was under way, Treasury agents guarded her.

2.8.2 The Second World War

When the United States entered the Second World War, the Coast Guard’s cryptanalytic unit, which she had built, was transferred to the Navy as part of OP-20-G. Its target became the clandestine radio networks of German intelligence in the Western Hemisphere, above all in South America. The NSA describes her wartime work as centring on German espionage communications from South America; her biographer Jason Fagone and the Coast Guard’s historians credit the unit with solving several Enigma machines used by German agents, and with exposing networks that the FBI later publicly claimed to have broken itself. The full record of the unit, filed under the designation OP-20-GU, remained classified until the early 2000s, and the detailed figures for the number of messages and networks vary between accounts. In 1944 her decipherments also helped convict Velvalee Dickinson, a New York antique-doll dealer who had passed information on U.S. warships to Japan in letters disguised as doll-shop correspondence. After the war she designed communications security systems for the International Monetary Fund.

Her work was almost invisible in public for decades after her death in 1980. The Coast Guard announced in 2020 that a new national security cutter would bear her name, and Fagone’s biography, The Woman Who Smashed Codes (2017), brought her story to a general audience. Volume 12 returns to why that recovery happened when it did.

2.9 What Riverbank Taught

Riverbank’s legacy to American cryptology came in four parts, and each is visible in the rest of this series.

A method. Friedman came to codebreaking from genetics, a science built on counting and on testing hypotheses against distributions, and he treated ciphers the same way. The Index of Coincidence is the emblem of that method, but the whole Riverbank series has its character: classify the system, measure what can be measured, test every guess against the data.

A literature. Before Riverbank, an American who wanted to learn cryptanalysis had Hitt’s ninety-page manual. After it there were eight pamphlets, which became the core of the Army’s training texts for the next twenty years (Volume 3).

A practice of teaching. The Friedmans wrote for students from the start, and taught classes before they had a government post. That habit — of writing down every solved problem as a lesson for the next person — is what allowed a handful of people in 1930 to turn into an organisation of thousands by 1945.

A partnership. Theirs was one of the most remarkable professional marriages of the century: two codebreakers who, by most accounts, rarely discussed their secret work with each other, and whose separate careers — his in the Army, hers in the Treasury and Coast Guard — together defined the American profession. Their shared headstone at Arlington carries a phrase from Bacon, the subject that had brought them together: “Knowledge is power” (Volume 12).

---

Next — Volume 3: 1930: Rowlett, Sinkov, Kullback and Hurt — Building the Signal Intelligence Service. A Virginia schoolteacher who could not find “cryptanalyst” in his dictionary, two mathematics teachers from Brooklyn, and a self-taught Japanese linguist; a locked vault full of the Black Chamber’s secrets; two years of problems before any real work; codes compiled by throwing index cards in the air; and the textbooks and correspondence courses that turned one man’s knowledge into an Army’s.