The Codebreakers — Volume 11 — Mechanising Cryptanalysis

11.1 About This Volume

Every technique in Volumes 6 to 8 is, at bottom, counting, sorting and comparing: tallying letters, sliding one text against another and counting coincidences, listing repeated sequences, matching cribs against ciphertext, chaining pairs into alphabets. The Signal Intelligence Service (SIS) did all of it by hand in 1930. By 1945 it was doing most of it by machine, on a scale that required hundreds of machines and more than a thousand operators, and the Navy’s codebreakers were doing the same. A few years after the war, the same organisations ordered some of the first stored-program electronic computers in the United States.

This volume follows that path. It describes what the punched-card machines were and how they came into the two services; what a typical run actually computed, demonstrated with a small simulation that sorts and lists real PURPLE traffic card by card; the failure of Vannevar Bush’s Comparator, the first attempt to build a machine specifically for cryptanalysis; the special-purpose machines of the war; and the turn to general-purpose computers that ended, for this story, with HARVEST in 1962. The main source is Colin Burke’s history for the National Security Agency’s Center for Cryptologic History, It Wasn’t All Magic: The Early Struggle to Automate Cryptanalysis, 1930s–1960s (2002; Library copy), supported by Samuel Snyder’s internal NSA history of its computers (1964). Page numbers are Burke’s printed pages.

The simulations are in 03-outputs/tools/tabulator.py, which checks itself in a seven-part self-test. They model what the machines computed, not how fast the real ones ran; where a speed is quoted it is one the sources give or imply, and the text says which.

11.2 Why Machines

The pressure was volume. Burke’s figures for the end of the war are startling. The Army’s codebreakers at Arlington Hall had “close to 400 IBM machines using a million IBM cards a day”; their tabulating workforce “grew from fewer than 100 at the end of 1942 to 600 a year later”, and by the end of the war “the SIS’s tab rooms had close to 1,200 workers”, many of them WACs on the night shifts (Burke 2002, pp. 130–131; Library copy). The Navy’s OP-20-G, after its move to Nebraska Avenue, became “one of the world’s largest users of IBM equipment. ‘G’s’ IBM machines were counted in the hundreds, and they used millions of punch cards a week” (pp. 74–75). Some 800 people in Washington worked on the Japanese Navy’s main code, JN-25, alone (p. 150).

Neither service started that way. Both started with a single analyst, a pencil and a sheet of cross-section paper.

11.3 The Navy Rents IBM, 1932

The Navy came first, and reluctantly. In September 1930 Captain Stanford Hooper urged OP-20-G to adopt formal methods and automation; the suggestion “was not too well received. OP-20-G’s principal civilian cryptanalyst, Agnes Meyer Driscoll, did not like the idea at all” (Burke, p. 17). What changed minds was a catastrophe. Around 1931 the Japanese Navy replaced the code the Americans had been reading — “seven years of work on the previous code had become valueless!” — with a new one, the Blue code, superenciphered by adding random numbers to the code groups. Over 100,000 words had to be decoded.

Lieutenant Joseph Wenger had surveyed the tabulating machines available. In early 1932 OP-20-G “decided to select the type centered upon electrical rather than mechanical reading of cards. Seeing Remington-Rand’s system as inflexible, they hurried to rent the electromechanical IBM tabulating devices, ones built to handle alphabetic characters as well as numbers. The punch card era seemed to have begun at OP-20-G” (pp. 18–19). The money was tiny: the Bureau of Engineering “was able to raise only a few hundred dollars, not several thousand”, a machine or two arrived, and their use “survived only as a near underground activity” (p. 19).

Burke’s judgement on what they were good for is precise. Tabulators “were well suited to many decoding procedures, especially those calling for sorting and, later, collating operations, but they were not the mathematical or truly high-speed statistical devices needed to break into the new cipher machines” (p. 19). That sentence describes the next decade.

11.4 Friedman Gets His Machines, 1934

William Friedman tried to acquire IBM machines for the SIS a few years after the Navy and “faced almost as many frustrations as Hooper and Wenger”, but “finally acquired some machines in late 1934. By 1937 he and his crew had developed several tabulator methods that became classic means of cryptanalytic attack” (Burke, pp. 22–23). Burke’s sources include Friedman’s “Addenda on the IBM Sorter” of about August 1935 and two papers of 14 April 1937 describing a cryptanalytic coincidence counter and an isomorph locator — the statistical and pattern-finding methods of Volumes 6 and 7, adapted to cards. Friedman and Rowlett also devised an attachment that made the tabulating equipment generate random code groups, for the Army’s own codebooks.

Snyder’s history counts about 750 punched-card machines installed in NSA’s predecessor organisations between about 1935 and the end of the war, and lists the five kinds and their jobs (Snyder 1964, pp. 3–4; Library copy):

Table 1 — Snyder's history counts about 750 punched-card machines installed in NSA's predecessor organisations between about 1935 and the end of the war, and lists the five kinds and their jobs ([Snyder 1964](https://media.defense.gov/2021/Jul/01/2002754530/-1/-1/0/6586784-HISTORY-OF-NSA-GENERAL-PURPOSE-ELECTRONIC-DIGITAL-COMPUTERS.PDF), pp. 3–4; [Library copy](/library/history-of-nsa-general-purpose-electronic-digital-computers-samuel-s-snyder-1964))

Machine	What it did
Keypunch	put the text on cards
Reproducer	copied and expanded card files
Sorter	distributed cards into pockets by the punches in one column
Collator	merged two sorted files and selected matching cards
Tabulator	counted, compared and printed

The machines were in the PURPLE work from the beginning. Friedman’s report of October 1940 thanks the tabulating supervisor, Ulrich J. Krapfl, and the cryptanalytic aide Mary J. Dunning for “countless tabulating machine operations”, and the card-punch operators for “painstaking work” (Friedman 1940). Rowlett recalled using IBM accounting machines years earlier to test RED traffic for repetitions.

11.5 What a Run Computed

A punched-card attack turned a question about text into a question about order. The commonest question was: where does the same sequence of letters occur twice? A repeated sequence in ciphertext might mean the same plaintext enciphered at the same point of the key — Kasiski’s clue — or two messages in depth. By hand, answering it meant reading every message against every other. By card, it meant four steps.

1. Punch. One card for every starting position in every message, carrying the four letters that begin there, the message number and the position.
2. Sort. A sorter reads one column at a time and drops each card into a pocket by the hole it finds. A letter in the Hollerith code is two holes — a zone punch in row 12, 11 or 0 and a digit punch in rows 1 to 9 — so an alphabetic column needs two passes, digit first and then zone. Four letters need eight passes, starting with the last letter, because each pass preserves the order left by the one before.
3. List. The tabulator reads the sorted deck and compares each card with the one before; when two consecutive cards carry the same four letters it prints them.
4. Read the list. The analyst examines each repeat.

The tool performs exactly these operations on six PURPLE messages: the real intercept of Part 1 of the Fourteen-Part Message and five messages of 1,200 letters enciphered on other keys. It punches 7,210 cards, sorts them with eight stable one-column passes — 57,680 card-handlings in all — and lists the repeats. The self-test confirms that the eight passes leave the deck in exact alphabetical order and that the listing finds exactly the repeats a modern dictionary lookup finds.

The listing contains 55 repeated four-letter groups. That is the interesting result, because it is the number Friedman’s analysts would have hated. For 7,210 four-letter groups drawn at random from a hat, the expected number of groups that occur twice or more is about 56. PURPLE’s ciphertext has no more repeats than random letters — no repeats caused by the plaintext — which is the phenomenon Friedman described in his report of 1940 (Volume 6). The same run on six Vigenère messages of the same English produces 1,030 repeated groups, almost all of them real. A card run could find Kasiski’s clue in a periodic cipher in an afternoon. Against PURPLE it found only chance.

What did it cost? Burke gives the design goal of Bush’s Comparator as 20,000 comparisons a minute, “forty times greater than an IBM sorter and 160 times faster than a tabulator”, which implies working rates of about 500 a minute for a sorter and 125 for a tabulator in this kind of work (pp. 36–39). Taking those as card-handling rates is an approximation, since a sorter’s mechanical speed and its useful rate in an analysis are not the same thing; on that basis the eight sorting passes take about 115 minutes of machine time and the listing about 58. Two or three hours, plus the punching — for a job that by hand would take days.

11.6 The Comparator

The other great question of Volumes 6 and 7 was are these two texts in depth? — the kappa test, sliding one message against another and counting coincidences at every offset. Cards did this badly. It is a comparison of everything with everything, and the number of comparisons grows with the product of the lengths.

In 1936 Wenger and Hooper turned to Vannevar Bush at MIT, who was then building the differential analyser and planning electronic calculators of his own. A Navy contract was signed in January 1937 (Burke, pp. 34–37). Bush and Wenger chose to automate “the heart of the IC method, coincidence counting”: two messages punched on paper tape, read photoelectrically, slid past each other at every offset while electronic counters totalled the coincidences. The target was 20,000 comparisons a minute. “Messages of 200 characters were typical … The very upper limit was 2,000 characters”, and a message of that length “called for almost 4,000,000” comparisons (pp. 36–39).

The tool shows what those numbers meant:

Table 2 — The tool shows what those numbers meant

Comparisons for two messages of…	Count	IBM tabulator (implied 125/min)	IBM sorter (implied 500/min)	Comparator goal (20,000/min)
200 letters	40,000	5.3 hours	1.3 hours	2 minutes
2,000 letters	4,000,000	533 hours	133 hours	3.3 hours

It also performs a run of its own. It takes the real Part 1 intercept and a modelled message of 1,000 letters enciphered on the same key and alphabet but starting 300 letters later — two messages that overlap in depth — and counts coincidences at every relative offset, 1,285,000 comparisons in all. The highest score is at an offset of +300: 73 coincidences in 985 letters, a rate of 0.074, almost six standard deviations above the random 1/26. The overlap is found. The runner-up, at an offset of 25, is the sixes period of Volume 6 showing through. A modern laptop performs the run in under half a second.

The Comparator itself was a failure, and Burke’s account of why is one of the most instructive stories in early computing. A team that included Jerry Jaeger, Richard Taylor, Herbert Grier and a machinist, Walter Kershner, built it under a manager who joined in September 1937. It reached Washington in late June 1938, a month late, and “would not start”: the manager had locked the punch and printer in a Cambridge safe-deposit box before leaving on his honeymoon. It was declared finished in November 1938, soon became inoperable again, and was “listed on OP-20-G’s equipment roster in 1939, but it was never used, not even on … the breaking of the Japanese Purple cipher machine”. A Navy engineer’s report found that “nothing but the electronic counters proved reliable”; the tape punch could not produce precise tapes. “In 1940, Safford … had to admit the machine never worked” (Burke, pp. 37–45).

Burke’s diagnosis is general. There was no viable memory technology; the components were hand-made; the design was fixed early and rushed to meet the contract’s deadline; and the data had to be entered on a punch that never worked. The electronics were the one part that was ready. The lesson — that a fast processor is useless without reliable input and storage — would be learned again, at greater cost, after the war.

11.7 The Machines of the War

From 1940 the Navy’s renewed effort, run by John Howard at MIT and then by Wenger and Howard Engstrom at OP-20-G, became what Burke calls “the foundation for the United States Navy’s incredible Rapid Machines Program” (p. 45). The Army, with fewer resources, built its own. The machines fall into three families.

Punched cards with relays. From early 1941 IBM added relays to its machines for the Navy “to strip superencipherments from the Japanese code and to flag repetitions of code groups”; by mid-1941 Thomas Watson had agreed to help, and the “Navy Change” machines followed — thirteen types, beginning with one that numbered cards automatically and an “automatic cross-footer” that “provided a high-speed means of decrypting additive cipher systems, such as those used by the Japanese Navy” (pp. 47, 51–52). The Army’s equivalent was a family of relay attachments, “some twenty-three different” ones against Japanese army systems, and the Slide Run of 1944, an IBM 405 tabulator joined to “a huge chest-high, multipanel cabinet full of advanced relay circuits” that held up to 1,500 code groups and read cards at 150 a minute (pp. 131–134). Leo Rosen, who built the PURPLE analog, built with IBM’s help the Gee Whizzer of spring 1941, a relay box with two six-level, 25-position rotary switches — PURPLE’s components again — that steered a tabulator’s scoring of trial solutions of transposition ciphers (pp. 50–51).

Film and photoelectric machines. Eastman Kodak built several for the Navy. Tessie, delivered in September 1942, searched 35 mm film for repeated four-letter groups — the repeat run above, on film. Icky counted coincidences on microfilm; Hypo, working by October 1943, supported Agnes Driscoll’s catalogue approach to naval Enigma. Copperhead scanned “one hundred message columns at a time” on 70 mm tape, and up to five were in operation by early 1945 (pp. 64–73).

Machines that imitated the enemy’s. The largest wartime programme was the Navy’s four-rotor bombe, designed by Joseph Desch at the National Cash Register Company in Dayton. Its first prototypes would not run reliably through mid-1943; in production it was a 5,000-pound machine housing sixteen four-rotor Enigma equivalents, and by mid-November 1943 Washington had more than fifty in operation, with over a hundred built in all — 121 is the figure most often given (pp. 107–111). The Army’s machines included Madame X; the Dudbuster, a relay device that recognised plain language; and a Purple Dudbuster for testing cribs. The PURPLE analogs themselves belong in this family.

The NSA’s own retrospective is exact about what these were not: “None of these machines, it should be noted, were computers. They had no memory, and … were ‘hard-wired’ to perform just one task” (NSA, “Before Super-Computers”, p. 1; Library copy).

PURPLE itself was mechanised once more, late. In April 1945 the Japanese changed the way PURPLE starting positions were chosen, so that each message had a new one; finding it by hand took about a week. An IBM card reproducer was connected to a PURPLE analog to punch and print all of the machine’s starting positions on cards. Making the file took about a day; with it, finding a message’s switch positions took fifteen minutes (Freeman, Sullivan & Weierud 2003, p. 30, citing the Signal Security Agency’s annual report for fiscal 1945; Library copy).

11.8 An Additive Run

The Navy’s largest card workload was not a cipher machine but a code. JN-25 was a book of five-digit groups, over 30,000 in the 1939 edition and over 50,000 later, superenciphered by adding five-digit numbers from an additive book of 300 pages of 100 numbers each. Its designers had built in an error check: every valid code group was divisible by three (Burke, pp. 150–152). That check was a gift to the codebreakers. If several messages had been enciphered with the same additive at the same point — in depth — then the right additive, stripped from each cipher group, must leave a group divisible by three in every case. The Navy built special machines for the test, among them a hand-cranked additive finder with fifty wheels and lamps that showed “over”, “under” or “divisible” (p. 52).

The tool reproduces the test with a synthetic codebook of groups divisible by three, a random additive and non-carrying addition, and asks how many of the 100,000 possible additives survive as the depth grows.

Table 3 — An Additive Run

Groups in depth	1	2	3	5	8	10	12	15	20
Additives left	33,334	13,343	5,472	1,281	73	22	13	1	1

With one group a third pass, as they must. Thereafter the survivors fall more slowly than a naive “one-third per group” would predict, because with non-carrying arithmetic the tests on different groups are not independent; it takes fifteen groups rather than eleven to single out the true additive. The true additive survives every test. This is the kind of computation the cross-footers and additive machines performed by the million: not cryptanalysis in the sense of insight, but the mechanical elimination that made insight affordable.

11.9 From Machines to Computers

The men who ran the Navy’s machine programme left the service at the end of the war but not the work. In January 1946 Engstrom, William Norris and their colleagues founded Engineering Research Associates in St. Paul, Minnesota, in a former glider factory, with financing arranged by John Parker; the Navy expected the firm to be its “captive” (Burke, pp. 219–220). After an OP-20-G officer, Lieutenant Commander James Pendergrass, reported on the Moore School lectures on electronic computers of the summer of 1946, Wenger ordered a general-purpose computer — project ATLAS, named after a comic-strip “mental giant” (pp. 240–241).

Snyder’s history gives the sequence (Snyder 1964, pp. 7–65; Library copy):

Table 4 — Snyder's history gives the sequence ([Snyder 1964](https://media.defense.gov/2021/Jul/01/2002754530/-1/-1/0/6586784-HISTORY-OF-NSA-GENERAL-PURPOSE-ELECTRONIC-DIGITAL-COMPUTERS.PDF), pp. 7–65; [Library copy](/library/history-of-nsa-general-purpose-electronic-digital-computers-samuel-s-snyder-1964))

Machine	Builder	Date	Notes
ATLAS I	ERA	delivered 8 December 1950	about 2,700 tubes; about $950,000; 16,384-word magnetic drum, 24-bit words; ERA’s commercial 1101 “almost identical”
ABNER	Army Security Agency with the National Bureau of Standards; Technitrol	completed September 1951; last instruction checked out April 1952	mercury delay-line memory; named after the comic strip Li’l Abner
ATLAS II	ERA	installed October 1953	commercial twin: the ERA 1103
IBM 701	IBM	delivered April 1953	one of IBM’s twenty “Defense Calculators”; Snyder’s table gives May
SOLO	Philco	delivered March 1958	transistorised copy of ATLAS II, $1,007,700; marketed as the Transac S-1000
HARVEST	IBM	delivered January 1962	about 15 months late and 32 per cent over budget; Snyder’s table gives February

The machines’ first jobs were the old ones. The NSA’s own history says that the first ATLAS program “is believed” to have attacked isologs — the same text in two systems — in the Soviet traffic known as VENONA (NSA, “Before Super-Computers”, p. 2).

HARVEST, the IBM 7950, was the culmination of the line. It was an extension of IBM’s STRETCH, with a stream unit for character processing and an automated tape library, TRACTOR, whose cartridges were fetched and mounted by machine: 1,128,000 characters a second per drive, up to 480 cartridges, about 44 billion characters on line (Snyder 1964, pp. 45–60). The National Cryptologic Museum calls TRACTOR “the first fully automated storage and retrieval system”, and an NSA evaluation quoted on its display card found HARVEST “more powerful than the best commercially available computer by a factor of from 50 to 200, depending on the particular application”. It ran from 1962 to 1976, when the NSA bought a Cray-1 supercomputer.

11.10 What Carried Over

The path from a rented tabulator in 1932 to HARVEST in 1962 is often told as a story about hardware. Read alongside Volumes 6 to 8 it is a story about a small set of operations that never changed. Count letters and coincidences. Sort, and list what repeats. Slide one text against another and score the fit. Test a hypothesis against every candidate and keep the survivors. Friedman’s analysts did these with pencils and strips of cross-section paper; the tab rooms did them with sorters and relay attachments; Tessie and Copperhead did them on film; ATLAS did them in a stored program. The operations came from the statistics and pattern methods of the 1920s and 1930s, and they defined what the machines were for.

The other thing that carried over was a hard-learned view of engineering. Bush’s Comparator failed for want of reliable input and storage, not for want of speed, and the people who watched it fail went on to commission magnetic drums, delay lines and automated tape libraries. Snyder, an interested party, argued that cryptologic customers pushed the computer industry towards exactly those features; whether or not that claim is accepted in full, the organisations that broke PURPLE and built SIGABA were among the first in the world to treat computation as an industrial process. Volume 12 asks what of all this is still taught.

---

Next — Volume 12: Why We Still Study Them. From Military Cryptanalysis to modern ciphers and the National Security Agency: what the school founded by the Friedmans left behind — in textbooks, in the design of ciphers that assume the enemy knows the machine, and in the people who learned from them.