Lorenz and Colossus — Volume 9 — How Colossus Worked — Tapes, Valves, Counters and the 1+2 Break-In

9.1 About This Volume

Volumes 7 and 8 told the story of the people and the building. This volume opens the racks. It describes what Colossus actually did with a message: how the tape was prepared and read, how the machine kept time, how it imitated the Lorenz wheels, how the logical test was set up, how the results were counted and printed, and how the cryptanalysts used a sequence of such runs to find the settings of the chi wheels.

The centre of the volume is Tutte’s 1+2 break-in, the statistical test that Colossus was built to run. It is described first in principle and then in practice, with a worked example: a real message, enciphered on a Lorenz SZ42 for the Cipher Challenge held at Bletchley Park in November 2007, run through the counting program written for this series (03-outputs/tools/colossus_count.py, described in Volume 14). That program does in software exactly the count Colossus did in valves.

The description is of the Mark 2, the production machine of which ten were built, except where the Mark 1 prototype differed. It draws on Copeland’s account, the wartime photographs in the General Report on Tunny, and the reconstruction work of Tony Sale and the rebuild team, who had to work out many of the circuits from first principles (Volume 11).

9.2 The Problem in One Line

Volume 3 describes the Lorenz SZ42 in full. For this volume one equation is enough. Each five-bit character of cipher text Z was the bitwise sum, modulo two, of three things:

Z = P ⊕ χ ⊕ ψ′

where P is the plaintext character, χ (chi) is the character formed by the five regularly stepping chi wheels, and ψ′ (psi-dash) is the character formed by the five psi wheels — written with a dash because the psi wheels did not step every time. Controlled by the two motor (mu) wheels, they all moved together or all stood still, so the psi stream contained runs of repeated characters.

Bletchley Park wrote the two binary values as dot (•, a zero, no hole in the tape) and cross (x, a one, a hole). Bits were numbered 1 to 5, called impulses, after the five signal elements of a teleprinter character. “Z1” means the first impulse of the cipher text; “χ2” the second chi wheel.

9.2.1 Delta

Tutte’s key move (Volume 5) was to work not with the characters themselves but with the delta of each stream: the sum of each character with the next one.

ΔZ(t) = Z(t) ⊕ Z(t+1)

A delta is a dot where two consecutive characters agree in that impulse and a cross where they differ. Taking the delta of the whole equation gives

ΔZ = ΔP ⊕ Δχ ⊕ Δψ′

and that is useful because of the psi wheels’ habit of standing still. Whenever they did not move between one character and the next, Δψ′ was all dots — the psi contribution vanished completely.

9.2.2 The 1+2 test

Tutte then added the first two impulses together:

ΔZ1 ⊕ ΔZ2 ⊕ Δχ1 ⊕ Δχ2 = (ΔP1 ⊕ ΔP2) ⊕ (Δψ′1 ⊕ Δψ′2)

The right-hand side is what is left over once the chi wheels have been removed. Two biases push it towards dot:

• Δψ′1 ⊕ Δψ′2 is a dot whenever the psi wheels stood still, and about half the time when they moved — Tutte’s figure was about 70 per cent dots overall.
• ΔP1 ⊕ ΔP2 is a dot rather more often than chance in German teleprinter traffic — about 60 per cent — because of repeated characters and the operators’ habit of doubling the shift and space characters.

Two biased terms added together give a weaker bias: the sum comes out at about 55 per cent dots (Copeland, Rutherford Journal 3; Wikipedia, “Cryptanalysis of the Lorenz cipher”).

But that is only true when the χ1 and χ2 streams on the left-hand side are the right ones — when the chi wheels are set at the starting positions actually used for the message. At a wrong setting, the chi terms do not cancel, and the left-hand side is effectively random: about 50 per cent dots.

So the test is: for every possible pair of starting positions of chi wheels 1 and 2 — 41 × 31 = 1,271 pairs — count the dots in ΔZ1 ⊕ ΔZ2 ⊕ Δχ1 ⊕ Δχ2 along the whole message. The pair with the highest count is probably right. Bletchley Park called this the 1+2 break-in, or the double-delta method. It assumes the chi wheel patterns are known, which by 1944 they usually were (wheel breaking is described below).

The bias is small. At 55 per cent against 50, a short message will not show it reliably; a message of several thousand characters will. That is why the method needed long messages and why it needed a machine.

9.3 The Machine, Unit by Unit

Wikipedia’s summary of the Mark 2, drawn from the wartime documentation, lists its main units as a tape transport with a photocell reader, a six-character shift register, twelve thyratron ring stores simulating the Lorenz wheels, switch panels for the program and for the “set total”, Boolean function units, a span counter, a master control, five electronic counters and an electric typewriter (Wikipedia, “Colossus computer”). The diagram shows how they connect.

9.3.1 The bedstead and the tape

The message was punched on standard five-hole teleprinter tape, and the Wren operators made it into a loop. Wikipedia’s account of the procedure, from the wartime documentation, describes them joining the ends with glue, leaving a blank leader of about 150 characters, and punching a start hole and a stop hole in an extra position, measured from the blank section and from the end of the message. The loop was threaded on the bedstead, a tall frame of pulleys that kept it under tension as it ran.

The tape ran past the reader at 5,000 characters per second. At ten characters to the inch, that is about 500 inches — some 40 feet — of paper every second, nearly thirty miles an hour. Flowers’ tests had taken the paper to about 9,700 characters a second before it disintegrated, so 5,000 left a margin.

The reader was optical. Light from a lamp shone through the tape, and a lens projected an enlarged image of the row of holes on to a mask in front of the photocells — one for each of the five data tracks, one for the sprocket track and others for the start and stop holes. Arnold Lynch designed it; Sale records that the photocells were vacuum types of a kind made for the proximity fuse, and that the reader is not shown in any surviving photograph, so the rebuild’s had to be reconstructed with Lynch’s help (Sale, codesandciphers.org.uk).

9.3.2 Time from the sprocket holes

Colossus had no free-running clock. Every teleprinter tape has a row of small sprocket holes, one per character, used to drive it through a mechanical reader. Colossus read them optically along with the data, and each sprocket pulse was the beat on which the machine acted. Sale describes the data being sampled on the trailing edge of the sprocket pulse and the logic result passed to the counters on the leading edge of the next. At 5,000 characters a second there were 200 microseconds between beats, and Sale estimated that the logic had time for a chain of up to about a hundred gate delays in each.

This was the simplest possible answer to Heath Robinson’s problem. If the tape slowed or speeded up, the machine slowed or speeded up with it. Nothing could get out of step with the tape, because nothing ran except at the tape’s command.

A second, slower cycle ran once per pass, triggered by the start and stop holes: it read out the counts, stepped the wheel rings to their next starting position, and reset the counters before the message came round again.

9.3.3 Staticisers, deltas and the shift register

The signals from the photocells were small and ragged. Staticisers sampled them and set them to clean standard voltages. A one-character delay — an integrating capacitor in the circuit Sale reconstructed — made the previous character available alongside the current one, so that the delta of each impulse could be formed by adding the two.

On the Mark 2 the delay was extended into a shift register: a chain of stores holding the last several characters read, each stepping one place along on every sprocket pulse. Sale noted that this may be the first recorded design of an electronic shift register. It served two purposes: forming deltas, and giving the machine a window of consecutive characters that five separate circuits could each look at differently (below).

9.3.4 Twelve thyratron rings

The heart of Flowers’ idea was that the Lorenz wheels did not need a tape. Each wheel was imitated by a ring of thyratrons, one valve for each position of the wheel: 41 for the first chi wheel, 31 for the second, and so on, 501 in all for the twelve wheels. At any moment exactly one thyratron in each ring was conducting, marking the wheel’s current position. On every sprocket pulse the “on” state passed to the next valve round the ring, as a wheel turns to its next cam.

The pattern — which positions of the wheel had a raised cam — was set by patch links, plugged on the ring’s panel to match the known wheel pattern. A uniselector, a Post Office rotary stepping switch, set the position at which the ring started each pass. The output of each ring was therefore the bit stream that wheel would have produced, starting at the chosen position. Sale records that the rings with an odd number of positions needed extra circuitry, and that the twelve ring control panels and the 501 thyratrons together occupied one rack of the machine (Sale, codesandciphers.org.uk).

In the rebuild team’s notes the chi rings are labelled K1 to K5; the cryptanalysts’ own notation used χ, ψ and μ for the chi, psi and motor wheels.

9.3.5 The selection panel and the logic

What Colossus did with the tape and the rings was set up by the operators on panels of switches, plugs and cords. The selection panel chose which streams went into the logic: the raw tape (Z) or its delta (ΔZ), the chi stream or its delta, the psi stream or its delta — and, on machines that could hold two tapes, whether the near or far tape was read.

The selected streams were combined by Boolean logic — sums modulo two, ANDs and ORs — and the conditions for each counter were set up on a large switch panel, the Q panel in Bletchley’s notation, together with a sloping plug panel. Sale gives the 1+2 test as an example of how a run was “programmed”: the delta outputs of tape impulses 1 and 2 were combined with the delta outputs of the first two chi rings, and the result routed to counter 1; a switch decided whether the counter counted dots or crosses.

This is the sense in which Colossus was programmable. Its program was not stored in memory, and it could not branch or loop by itself; but the same hardware could be made to count any one of a very large family of Boolean functions of the tape and the wheels, by setting switches and plugs. Wikipedia’s account cites an estimate of some five billion possible combinations of the main switch panel and plugboard. In practice, the Newmanry used a well-worked repertoire of runs.

9.3.6 Five counters

Each of the five counters was a chain of electronic decade counters, descended from Wynn-Williams’s design for Heath Robinson (Volume 7), able to count to 9,999. On each sprocket pulse, a counter added one if its condition was met. At the end of each pass — signalled by the stop hole — the counts were transferred to a bank of latching relays, the counters were reset, and while the tape made its next pass the relays drove an electric typewriter that printed the previous pass’s counts and the wheel positions they belonged to.

A span counter allowed counting to be restricted to part of the tape. If a stretch of the intercept had been badly received — fading on the radio link was common — the operator could set Colossus to ignore it, so that garbage did not dilute the statistic. This was the “spanning” ability that the first Heath Robinson had lacked.

9.3.7 The set total

Most of the 1,271 counts in a 1+2 run were of no interest. Printing all of them would have wasted time and buried the one that mattered. Colossus therefore had a set total: a threshold, set on banks of decade switches, below which a count was not printed. The cryptanalyst chose it from the length of the message, so that only counts well above the chance level — the candidates — appeared on the typewriter.

9.3.8 Five tests at once

The Mark 1 tested one pair of wheel starting positions per pass of the tape. A full 1+2 run therefore took 1,271 passes. The Mark 2 used its shift register to do five tests per pass. Five identical logic circuits looked at the tape through the shift register at five consecutive positions, while the wheel rings ran once; each circuit was therefore, in effect, comparing the message with the chi wheels started one position further on, and each fed its own counter. One pass of the tape scored five neighbouring settings at once, cutting a full run from 1,271 passes to about 255. At 5,000 characters a second the effective speed was 25,000 — what would now be called parallel processing, by a machine of 1944.

9.4 Running a Message

A run, in outline, went like this.

1. A message arrived from the intercept station as a punched tape (Volume 10). The Wrens made a loop of it, with its leader and start and stop holes.
2. The cryptanalyst on duty decided which run to do first — almost always the 1+2 run — and gave the operators the settings: which streams to select, the logic for each counter, the starting positions of the rings, the set total.
3. The operators set the switches and plugs, threaded the loop, started the tape motor and then the machine.
4. The tape ran round. On each pass the counts above the set total were printed, with the ring positions; the rings stepped on; the tape came round again.
5. When the run was finished, the cryptanalyst read the printout, picked the best candidate, and specified the next run.

Wikipedia’s summary of the wartime timings is that a long run — one that stepped two wheels through all their combinations — took about eight minutes on a Mark 2 for an average message, and a short one-wheel run about two minutes.

9.4.1 A 1+2 run, worked

The following numbers come from this series’ counting program, run on a real message: one of the three transmissions sent from the Heinz Nixdorf MuseumsForum in Paderborn during the Cipher Challenge of 15–16 November 2007 — the one logged as 16:00 UTC — enciphered on an original Lorenz SZ42 for the Cipher Challenge that marked the completion of the Colossus rebuild (Volume 11). The cipher tape used is the one recorded and published by the challenge’s winner, Joachim Schüth. The program was given only the cipher text and the chi wheel patterns, which the challenge had published — exactly what a Newmanry cryptanalyst would have had. Its self-test checks that it recovers the chi settings that Schüth published for all three challenge messages; it does.

The message is 5,880 characters long, giving 5,879 deltas. The program counts, for each of the 1,271 starting pairs, the positions at which ΔZ1 ⊕ ΔZ2 ⊕ Δχ1 ⊕ Δχ2 is a dot:

Table 1 — The message is 5,880 characters long, giving 5,879 deltas. The program counts, for each of the 1,271 starting pairs, the positions at which ΔZ1 ⊕ ΔZ2 ⊕ Δχ1 ⊕ Δχ2 is a dot

Rank	Count of dots	χ1 start	χ2 start	Standard deviations above chance
1	3,286	36	27	9.0
2	3,110	36	19	4.4
3	3,087	36	4	3.8
4	3,068	26	27	3.4
5	3,062	35	31	3.2
6	3,060	34	27	3.1

Source: colossus_count.py, Lorenz-and-Colossus tools, run 27 October 2026 on the 2007 Cipher Challenge 16:00 UTC tape. Wheel positions are numbered from 1 as in the program; wartime numbering conventions differed.

At a wrong setting the expected count is half the number of deltas — 2,940 — with a standard deviation of about 38. The top count, 3,286, is about 55.9 per cent dots, very close to Tutte’s 55 per cent, and stands nine standard deviations above chance. The next best is less than half as far out. Several of the runners-up share one wheel position with the winner (χ1 at 36, or χ2 at 27), which is what one would expect: getting one wheel right cancels part of the chi contribution and leaves a little of the bias.

A Colossus with its set total at, say, 3,100 for this message would have printed only the top two lines, and the cryptanalyst would have read off χ1 = 36, χ2 = 27.

9.4.2 The other three chi wheels

With χ1 and χ2 set, the remaining chi wheels were set by shorter runs, each stepping one wheel through its positions with the others held. The Newmanry’s actual sequence of runs varied with the message and is described in detail in the General Report on Tunny; the simplified version used in this series’ program combines the counts for “1+k” and “2+k” for each remaining wheel k. On the 2007 message it sets χ3 = 2 (4.4 standard deviations above chance, against 2.6 for the runner-up), χ4 = 1 (6.8 against 3.0) and χ5 = 23 (10.7 against 4.9) — again matching Schüth’s published solution.

The final step, which Copeland describes as an acid test, was to check that the de-chi — the cipher text with the chi stream removed — had the statistical look of plaintext plus a stuttering psi stream. On Colossus this, too, was a count.

9.5 What Colossus Did Not Do

At the end of a chi-setting run the message was still unread. Colossus had produced five numbers: the starting positions of the five chi wheels. The rest of the key — the psi wheels and the motor wheels — had to be set before the plaintext appeared.

That was the Testery’s job (Volume 6). The Newmanry sent over the de-chi, a tape of the cipher text with the chi stream removed, which was plaintext plus the psi stream; the Testery’s cryptanalysts set the psi and motor wheels by hand, using their knowledge of German military language and teleprinter conventions. The final plaintext was produced on British Tunny machines — electromechanical analogues of the Lorenz machine built at Dollis Hill — set up with all twelve wheels. Colossus itself never printed a word of German.

9.6 Breaking Wheels

Everything above assumes that the chi wheel patterns were known. The Germans changed the patterns periodically — at first monthly or less often, for some links — and each change meant that the patterns had to be found again from the traffic before setting could resume. This was wheel breaking, and until 1944 it was done by hand, by a method developed from Turing’s work in 1942, known as rectangling: a large tabulation of the delta cipher text, arranged by the positions of two chi wheels, from which the patterns could be inferred statistically.

In February 1944 Donald Michie and Jack Good worked out how Colossus could help, and Colossus II was built with the capability (Copeland). Wikipedia records that Colossi 2, 4, 6, 7 and 9 carried a “gadget” to aid rectangling. Of the ten machines at the end of the war, seven were used for wheel setting and three for wheel breaking. The capability became critical in the summer of 1944, when the Germans began changing the wheel patterns daily — far faster than the hand methods could keep up with (Volume 10).

9.7 Colossus in Numbers

Table 2 — Colossus in Numbers

Feature	Mark 1 (1944)	Mark 2 (1944–45)
Valves	about 1,600 (Copeland); about 1,500 (Sale)	about 2,400 (Copeland); 2,500 (Sale)
Tape speed	5,000 characters/s	5,000 characters/s
Effective speed	5,000 characters/s	25,000 characters/s (five-way parallel)
Counters	not reliably recorded; one setting tested per pass	five, each to 9,999
Wheel rings	chi rings for setting runs	twelve rings, 501 thyratrons
First in service	5 February 1944	1 June 1944
Number built	1 (later upgraded)	9 more by May 1945

Sources: Copeland (ed.), Colossus (2006) and colossus-computer.com; T. Sale, codesandciphers.org.uk; Wikipedia, “Colossus computer”. The Mark 1’s exact configuration is less well recorded than the Mark 2’s.

9.8 Why It Was Fast Enough

The arithmetic is worth doing roughly. The 2007 message is 5,880 characters, so one pass of the tape at 5,000 characters a second takes about 1.2 seconds, plus a little for the blank leader. A full 1+2 run on a Mark 1 needed 1,271 passes — about half an hour. A Mark 2, doing five settings per pass, needed about 255 passes — five or six minutes — consistent with the “about eight minutes” Wikipedia gives for an average long run. The follow-on runs were shorter. A message’s chi setting could be found in well under an hour, instead of the centuries that Newman had estimated for hand methods.

The fastest modern equivalent, Schüth’s laptop program in 2007, did the same count in well under a minute (Volume 11). The Colossi were not fast by modern standards. They were fast enough, by about four orders of magnitude, to turn Tutte’s method from an impossibility into a production line.

---

Next — Volume 10: Fish at War — Tunny Links, Sturgeon and D-Day. What Colossus was reading: the radio teleprinter links between Hitler’s headquarters and his army groups, named by Bletchley after fish; the intercept station at Knockholt and the slip readers who turned ink traces into tape; the Siemens T52 — Sturgeon — which was a different machine altogether; Kursk in 1943; and the messages that told the Allies, in June 1944, that the deception over the invasion of France had worked.