Historical Document · 2002
NSA CCH history of US cryptanalytic machinery: OP-20-G/SIS IBM tabulators, Bush's Comparator and the RAM program, Navy/Army bombes, Copperhead, Slide Run, ERA, Atlas, Abner, Harvest (362 pp, 48 MB, text layer present)
Description of document: United States Cryptologic History, Special Series, Volume 6, It Wasn’t All Magic: The Early Struggle to Automate Cryptanalysis, 1930s – 1960s, Colin B. Burke, Center For Cryptologic History, National Security Agency, 2002 Requested date: 04-August-2012 Released date: 29-May-2013 Posted date: 24-June-2013 Source of document: National Security Agency Attn: FOIA/PA Office (DJ4) 9800 Savage Road, Suite 6248 Ft. George G. Meade, MD 20755-6248 Fax: 443-479-3612 (ATTN: FOIA/PA Office) On-Line Request Form The governmentattic.org web site (“the site”) is noncommercial and free to the public. The site and materials made available on the site, such as this file, are for reference only. The governmentattic.org web site and its principals have made every effort to make this information as complete and as accurate as possible, however, there may be mistakes and omissions, both typographical and in content. The governmentattic.org web site and its principals shall have neither liability nor responsibility to any person or entity with respect to any loss or damage caused, or alleged to have been caused, directly or indirectly, by the information provided on the governmentattic.org web site or in this file. The public records published on the site were obtained from government agencies using proper legal channels. Each document is identified as to the source. Any concerns about the contents of the site should be directed to the agency originating the document in question. GovernmentAttic.org is not responsible for the contents of documents published on the website. NATIONAL SECURITY AGENCY CENTRAL SECURITY SERVICE FORT GEORGE G. MEADE, MARYLAND 20755-6000 Serial: MDR-68543 29 May 2013 In response to your 4 August 2012 declassification request, we have reviewed the NSA cryptologic history entitled: It Wasn't All Magic: The Early Struggle to Automate Cryptanalysis, 1930s-1960s. The report has been reviewed under the Mandatory Declassification Review (MDR) requirements of Executive Order (E.O.) 13526 and is enclosed. Some portions deleted from the document were found to be currently and properly classified in accordance with E.O. 13526. The information denied meets the criteria for classification as set forth in Section 1.4 subparagraphs ( c) and (d) and remains classified TOP SECRET and SECRET per Section 1.2 of E.O. 13526. The withheld information is exempt from automatic declassification in accordance with Section 3.3(b) (2) and (3) and (h) (6) of the Executive Order. Section 3.5 (c) ofE.O. 13526, allows for the protection afforded to information under the provisions of law. Therefore, information that would reveal NSA/CSS functions and activities has been protected in accordance with Section 6, Public Law 86-36 (50 U.S. Code 402 note). Since your request for declassification has been denied in part, you may file an appeal to the NSA/CSS MDR Appeal Authority. The appeal must be postmarked no later than 60 calendar days after the date of the denial letter. The appeal shall be in writing addressed to the NSA/CSS MDR Appeal Authority (DJS), National Security Agency, 9800 Savage Road, STE 6881, Ft George G. Meade, MD 20755-6881. The appeal shall reference the initial denial of access and shall contain, in sufficient detail and particularity, the grounds upon which the requester believes the release of information is required. The NSA/CSS MDR Appeal Authority will endeavor to respond to the appeal within 60 working days after receipt of the appeal. Sincerely, BLAKE C. BARNES Chief Declassification Services Encl: a/s Cover Photos: (U) (clockwise from upper left): Hypo, Vannevar Bush, Harvest, Joseph Desch, Gee Whizzer, Stanford Hooper, Bombe, Samuel Snyder T9P 6E6RE'FN69MIN'Ft:REL l9 YBA; MfB, 61'cN, SBR ilcNB Nilh1*1 ii ~ Ii i i II II I~ UNITFV STATFS Ct?YPTOL<!JtiIC HISTORY spec£a!,Ser~ V~6 IC-Wa,m/C-AlbMagto: TJie, Early SCr~ Co-A ut-o-ma@ CrypCanaly~ 1930,f' - 1960y ~ II I I I I I Colin B. Burke ~ ~ ~ I I I I I~ I i I I CENTER FOR CRYPTOLOGIC HISTORY NATIONAL SECURITY AGENCY 2002 mp SFCRETl/COMIMt''~liL. TQ YG.t., i\YS, 61'cN, SBR ANB N!LJ1XI I~ ~ ii ~ ~ II II I~ ~ I~ I I I I I I ~ I i I I I II I I I~ This page intentionally left blank Page ii feP SEeKEl'llCOMINTflM!t TO t:ISA; AtrS, CAN, GBR AND NZDIX1 CONTENTS . P(lge (U) Forew01·d • . • • ••..•••.• •_ • •-• • -• ·--• -• • • _. _ -•: •-._-. -• • -••:-_• •- •-·---.••--·---·-·-·--·-XI (U) Introduction (U) Before NSA Openedlts Files ..................... • .••.• .1 (U) The Stocks J.\Tere Not All They ••.••••••••••••••• • •.•••• •1 (U) Ask H7hat the Agency Didfor ..... ·• .•..• • ••..••••••••..••2 (U) Ask Not H7hat the Agem.;y Can Do For . . • • • • . . • • • •.•.• .3 (U) Inspiration Patience ............................. .4 (U) There Wasn't Enough Magic ..••••• ·•· •....•.••..•..•.• •5 (U) A Story ivith Only a Few Ac1·.onyms ................. •5 (U) Two Decades befQre the Computer ..................... .6 and Chapter 1 (U) AnAcademic in Need of the Navy ... Until . (U) Anlnstitutionfor th:e Real World •..••. • ................ .7 (lI) A Manfor All Technologies •· • • . . . • . . • • • • • • . • • • • . ••••• .8 (U) More Than anlngenious Yankee ·•~ •.••••..••.•...••••.•.•• •9 (U) ThePQlitics ofMathematics and Engineering ..•.••..•.•• . 9 (U) The Manager ofScience ••..•••••••..••••••••.•• ·• •••• •10 (U) Bush and Stratton;'s Dream •.•.•••.•.•.••. • ••...•.. • •••• 10 (U) Bush Con.fronts Little Science •....••••••..•••.•.•• ~ .• •10 (U) Bush's GJ•eat Pla;ll ............................. ~ ..•••.• •u (U) Beyond .Analog Mechanicalllfachines •...••••••••••••• •11 (lI) J_wo Men with _a_ JVeed-- '! • ~- _._ ._ • _• • • •- • • • • -•-=- ·-- •• •: • • • - _•• -• • • •- •- •• • J:_2 (U) AMan for the Ngvy ......... ·•• ••..••••••..•..•.•• u •• •13 (U) Another PlanforScience and the Navy •••••••.•••• •..• •14 (U) Hooper Confronts the Bureaucr(lcy • .; .•...•.••. n • • • • • • 15 (U) A Few Men and Women for Secrecy ...•• • •••.•••.•••• •1(j (U) The Searchfor Pure Cryptanalysis .· • ..••••••.••••••.... •16 (U) From Electronics to Electromechanics .••.•.•...•••••••. •18 (lI) A Young Man/or the Future .•••.•.•.•.•.•••.••.•..•••. .ao (U) The Dream Postponed Again •• ·• ...•.••.•••.•...••...•.•20 (U) The Dreum Rebor.n, for a Moment •• q • • • • • • • • • • • • • • • • • 21 (U) Little Science Meets the Little Nuvy, Again ...••.•••..•• .21 (U) A JYIan for Statistics •••.•••••..•.•.• ·• .•••.••••••••...••22 (U) Science andthe Navy Need Other Friends .••••••••••••••23 (U) The Private World ofScience •••••.•..•.••• ·• ••..•••••••24 (U) A Man for Applied Mathematics and Information . ••• ••••24 (U) American Science and the War - the NDRC •.•• n • • • • • •24 feP 8E8RE'fh'88MIN'FllREL 'Fe tl9A; Al::l8, eitcN, 8BR ANB Ni!!b'IM1 Page iii T6P SEeRETHeeMINTflREL Te l:JSA; At:JS, eAN, 8BR ANB NZL/flt1 (U) Corporate Charity .................................. .25 (U) The Navy Comes in Second ....•.............•.•...... 26 Chapter 2 (U) The First Electronic Computer: Perhaps (U) A Reminder ofHooper's Hopes and Frustrations ...•.. •33 (U) The lnstitutiona.l Context ....................••..... .33 (U) The First Defeat: Bush Is Rejected ..•..••..•...•...•. .34 ([;J A Machine Too Soon ............................... .34 (U) The Decision to Build a Machine ...•.....•.••.....•.• •35 (U) Bush and Wenger Select a Problem •.•..••.•......... •35 (U) The l'f!..dex •....•••..••.....•...••.•....•.•... ·• ..•.. .36 (U) An Added Bonus, Possibly ..•.......•..•..••.•••.••.. •37 (U) Bush Outlines .the Machine and Sets Different Goals .•.. .37 (U) The Comparator Really Doesn't· Go to Washington .••.. .38 (U) Too Much to Ask ofMere Machines •.•.........•••••.. .39 (U) No Thanks for the Memories ......................... 40 (U) The Limits ofMechanics ...•.....•..•.....•.•..••.•• .40 (U) Let There Be Light, But Not Too Much ....••.•••..•... .41 (U) The Most Difficult Problem ofAll, But It Wasn't ••••...• .41 (U) The Easiest Becomes the Most Dij}icult ...........•.•..42 (U) Beyond Murphy's Law ••.•.•••••...••••....•.••...•••42 (U) Spring Is a Time for Love, Not Machinery ..........•. .44 (U) RAM Project Seems to Die, Late 1938 ••••••••••••••••• •44 (U) A Comparator There May Never Be .••.•.......•...•. •45 (U) Big Science Begins to Emerge .....•....••.••......••. .45 (U) Fire Cont:rol ......................•.............•... .46 (U) The Second Comparator .•..•••...........•.....•.•. .46 (lI) OP-20-G and Ultra ............................ ! • • • • • 47 (U) So Long for So Little ....•...••........•••.....•.... .48 (U) The Search for the Second Comparator ..••.........•• .48 (U) No Eqµal Pm·tners in Ultra ...•...•.....•.....••.•..• .49 (U) Another Machine That Wouldn't· •••.•.•••.•.•..•...•. •49 (U) The Revenge ofMechanics: the First Rounds .••..•.••. .50 (FS/;'81//R.l!sb) Logs and Relays - the Gee l1lhize1·. . ..•..••. .50 (U) The Navy Get Some Changes •.••.........•.•.••..•.•. .51 (U) The Greatest Kludge ofAll, But It Worked ...•.•••..... •52 (U) Trying to Save Bush's Reputation ....•.•...•.•...•... .52 (U) Yet Another Chance ................... ·· ...... • ..... .52 (U) lVhen the Ciphers Can't Be Broken .•.•.•..••..•..... • .53 (U) Wenger to the Rescue ......•.•...••.....•.•.••••••• •54 (U) Mathematics to Meet the Great Challenge .......•..•.. •54 (U) Bureaucracy vs. Science, Again ••..•••••••.••••.•.•.. •54 (U) A Seeming Victory for Science .•.•••.•...•••....•...• .55 Page iv i:gp SESRElfl69MINl'llREL "f0 l:ISA; Al:IS, eAN, e!" AHO 14lUlX I T0P 9EeREli'l00MIN'f/IREL :re l:l6ili; Al:IS, SAN, SBR ANB N26'i'X1 Chapter 3 (U) Bush's Dream Does Not C01ne True (U) A Look Ahead to Peace. . ...••••.••••••.• • •••••••••••. 63 (U) January 1942: Too Much Too Late •••••••.••••••••••• •63 (U) A Giant Step Backwards ••...•.•.••..•••......•...••. 64 (U) Haste and Confustion ••.••••..........••.•...•....• . 64 (U) Tessie nrouldn't Either ....••••••••.••...•.••••••.••. •65 (U) Tessie-'s New Hat .••.....•••••.••..•.•...•••.•..•..•• 66 (U) You Can Use Some ofthe Technology Some of the Time, But.• ....•...•......•.......•...... 67 (U) A Machine for Mrs. Driscoll's Special Problem ••••••.•. 69 (U) A Paper War, Perhaps ••••..•.•.•...•..••••••.•••••.• 71 (U) The Comparator Dies, Again .........•..••..•••.•••••. 71 (U) Almost Another Digital Machine •..•...•..•...•..••••• 72 (U) The Old Technologies Are the Best Technologies, for a Time ••••....•••.......•.•.......•.•......•.. .74 (U) Meanwhile, the Tabulator's Revenge .•.••.••••••..••• •74 (U) IBM's Most Special Contribution ...•••••......••••••• .75 (U) In the Absence of Rapid Ma.chines ......••••..•...•..•• 77 Chapter 4 (U) lUeeting the Crisis: Ultra and the Bombe (U) Looking Ahead - Ultra Saves RAM and OP;.20-G Creates a Science Company ...••.•.•....••..••.•••.•• .83 (U) The ''E'-' J\rfachine .................................... •83 (U) Only a Few Were Able and lVilling to Tackle ''E" •.....•. 84 (U) The Poles Automate Cryptanalysis in Their Special lit.Tay ••••••••••••••••••••••••••••••• •85 (U) Keeping the Bombe Secret for Too Long ••••.••••.•••• •85 (U) A Fresh Start against "E" •••.•...•.•••.•...•.•..••.•• 86 (U) Analog and Parallel May Be Fa.'tt, But •••••••.•.••.•.•• 88 (U) Ask and Then Not Receive •.•.•..••••••.•.•••.....•••. 89 (tJ) Ga1'e All and Get ..•...•••...•••••••...•••.•••.••..• . 90 (U) l1'hat Happened After ••....•••.....••••••..••••...• . 91 (U) Trust Builds Very Slowly .•...••.•••..•••...•.•••..• •91 (U) Agreements and Agreements and Agreements, But ••••• . 91 (U) Going Separate nrays ••.•..••.•.•.•••..••••.••••...• 92 (U) America without an Ultra .•.•••....•••.•.•••••••.••• .93 (l!) An American Ultra, Perhaps •..•..•....••••..••••••• •93 (U) Faster Than a Speeding Relay •••••..•••••••••••..••• •94 (U) Great British Expectations .••...•••..•••••.••••••••. . 94 (U) Great American Expectations •..•...••...•.•.•••.••. . 94 (U) Trying to Step Forward, Not Back .•...•..•.••.••.•... 97 (U) Britain's Own Version ofBush's Electronic Dreanis •••. •97 ;gp Sl!GREl;tJSQMINl=NREL 1=9 l:ISA, Al:IS, SAN, SBR ANB N:i!!UIM1 Page v TeP 9EeRETN80MINTllREL Te YSA; M:IS, SAN, 9BR AN9 Hl6'1*1 (U) No Time for Electronic ..•....•....•....•..•........ . 98 (U) A Crisis ofOrganizationa and Technology ..••.•.....• . 99 (U) Searching for a Place in Ultra ..••••.......•••...•••. . 99 (U) The Power ofInnocence ....•...•.•..•..•........... . 99 (U) The Power ofIgnorance ..•..•...........•...•..•... .i.01 (U) The Cousins Will Have Their Way ..• to aDegree .•.•••. .i.03 (U) A Long Apprenticeship .....•..•...•.......•...•... .104 (U) Desch Takes Charge .........•....•...•.••...••...• .105 (U) Wenger Gets His Organization •..••...•.••......... •105 (U) Of Tires und Transmissions and u Disappearing Laboratory .......••..•............ .i.05 (U) Saving the American Bombe •.•...•.......••..•..•. .io8 (U) A Bomb~ ~Too Lq.te •..•...•..••••.•..••....••...•••. -!-OB (U) A Progrum Bused on Another Technological Bet ...... .109 (U) July 26th: a Day of Defeat •..•.•.•.•...••.••••••... .109 (U) A Victory a Bit 100 Late •..•••.•...•..••..•......•. .i.09 (CJ) Ignorant No More •..•.•...• ~ ...•••.••.•.••••••.••• •1i1 (U) The Bombes at Work ..••...•.....•..••............. .ni (U) More to It Than the Bombe ..•........•.•..••.......•u2 Chapter .5 (U) A Search for Other '"Bombes" (U) Meanwhile, the Army .•........••.......•. • ....•....i.27 (U) The Search for Another American mtra ...•.•...•...• .128 (U) A Great Electronic Adventure, the Freak ••......••.•..i.29 (U) Tabulators and Truffic: A Data Processing War •.•..• .i.30 (U) Making the Tools More Powerful ........•........•.. •i.32 (U) Slides, Runs, and Endless Decks of Cards ........•... •i.32 (U) The Other Bombe Program .......••••...•..••••.••• .i.35 (U) Another Step Buck .................•..•..•.•....•• .135 (U) More to It Than the Machine ..•..•...•...•...••••... .137 (U) A Machine Looking fo1· Work .....••......•...•..... .i.39 (U) More Emergencies and More Comp1•omises .•...•..•. •i.40 (U) The 'Otlter ,PurpleS •.••.•..•••.••.•••••••••••••...•• •142 (U) New Guys and Old Guys, New Techniques and Old Insights . •.-• .•.•..•....• ~ •••....•....•.•••. •142 (U) A Matter ofMuchines and Control ..•.•.••.•..•..•.. . 143 (U) The Snuke Thut Died Too Young, Viper .•.•..•••..... .144 (U) A Snake in Hand, Perhaps - Python ..•.••...•.....•. .i.46 (U) OfStrips and Strippers .....•.............••....... .i.48 (U) Strips Without Strippers • ..•......•..••.•••••••••.. •149 (U) The Attack on the ftfany JN25s ....•....•.•...........1,50 (U) The Comparators That HTeren't the Copperhead Proposuls · und the Victory ofElectJ•onics ......•..•••. •153 Page vi TQP GESRETHSQMIN"FllREL T9 1:16A; AY6, SAN, 98R ANB NZL/JM1 1'9P SESRE"FN69MIN"FNREl 1'9 l:ISA; itil:IS, SAN, 08R ANB Ni!l/i'X4 (U) Beyond the Copperheads - the JN25 Crisis and ''M's'' Response ••••••••••••••••••••••••••••••• •155 Chapter 6 (U) Beyond the Bombes and Beyond World lVar II (CI) After the Bombe ••••••••.•••••••••••••.•••.•••••••• •-i.63 (U) Every tt'hich Way: The Code Challenge Continues •••• •164 (U) The Navy's Madame X - the Strongest Selecton ••.••.• •165 (l.I) A Wall ofKnobs .••••.••.••.••••••••••••••••••••••••166 (LT) Walls ofTubes •••••••••••••••••••••••••••••••••• -· • •167 (U) Into the Beyond and the Past, Rooms of Wires and Disk ••••••••••••••••••••••••••••••••••••••••• •168 (U) Desparate Options and a Conservative Selector ...•.•• •168 (U) Walls ofPipes and Thousands ofDots ••.•..••••••.•• •169 (U) The Relay Selector Gets an Electronic Face Lift ••••••• •171 (U) The Biggest Snakes ofAll - The Navy Almost Builds an Electronic Bombe ••••••••.••••••••••••.•.••••••• •173 (U) The Serpent and Friends .•...•.•.•.•..•••...••••••• •175 (U) The Revenge of the Enigma - or Electronics Is Inesca,pable ••••.•••.•••••••••...•.••••••• • ••••• •:177 (U) Beyond Cribs: the Statistical Bombe .•••••.••••••... .178 (U) No Escaping Electronics, Enigma ftfeets the Cobra •••• •180 (U) The Navy·'s Duenna •••••••••••••••••••••••••••••.• •i8o (U) From Relays to Tubes, Rosen Gets His Chance ..••.... .182 (U) Engineering Pride and Peacetime Priorities .•.•.•...• •183 (U) Keeping the Faith: the Return of the Film Machines •• •184 (U) 1'he Revenge of the Codes, Again •..•.....•••••.•.••. .184 (U) More Numbers Than Ever Before •.••••..•.•••.•.••. •186 (U) Dr. Bush, Your Best Friend Is Really the Army ••••••• •189 (U) The Great 5202 ••••••••••••••••••••••••••••••••••• •:190 (U) Beyond the Comparators ••..•....•.•••••••••...•••• .191 (U) The Machine That Wasn't ...•...•.•.•••.••••••••.•. •191 Chapter 7 (U) The Magic Continues (U) lFould History Repeat Itselfl ••••..••.•••.••..•••••• •199 (ll) l111tat There l1lasn 't . . . ....•........................200 (U) Signs ofSome Appreciation ••.....•••.•..••••..•.•• .201 (U) More MAGIC: Cryptanalysis Continues as Before •••.. 202 (U) A Cryptologic Future: Architecture and Ambiguity and Budgets •.•..•.•.....•..•..••••..........•.••. .203 1e Think): (U) The Enigma Is Dead Long Live the ............. ~. ~.. ~.••.••....•.••.•••.•.• •204 (U) A Hangover onl Anot er Time .... ~ ~~ ~ • • •••••• ~. ~ . •205 ···P.L 86-36 mp SliCRET"CQ~~l~l'fl/REb Te l:ISA; Al:IS, 6AN, SBR ANB NZUl)(1 Page vii T9P SE6RETN69MINTNREL 1'0 l:ISA; Al:IS, SAN, 8BR ANB NZL/IM1 P.L. 86-36 (U) Mrs. O'Malley's. Wayward Son . ~ ••• ~ ~ ~ •....••...•.• .206 f§//SI) The Grand !Machine .....•...•...•.. .208 (U) Hecate's Impressive Competitor ..•.••..••..••••...•• .211 (U) The Universal RAllfs ••.•••. ·••••••••••••••••.••• " •.• •21:1 (U) The Illusive Mat:rix .•..........•. -.- .•........ ~ -• .•-... •2i2 (U) It's a Nice Idea, Dr. von Neuman, But .....••.....•... •212 (U) Faith Without Institutions: Slides, Sleds and Skates .. •214 (U) Faith and an Institution: the Chance to Begin an Era . .218 (U) A Bright Hope for llooper's Dream •.....•.....•.... •220 (U) 1'he Grand Machine of Its Time, the New Comparator • •221 (U) Meanwhile, a Last Chance for Microfilm .•...•..•••.• .223 (U) Finally, the Electronic Bombe ......•..••.•.•..•.•.. .223 I Chapter 8 (U) Courage and Chaos: SIGINT and the Computer Revolution (U) It Wasn't Safe at the Cutting Edge .•.....•...•.••..• .233 (U) An Idea DWered ••_••••••••••••••••••••••••••••.••• •233 (U) Goodbye Dr. Bush, Hello Professor von Neuman ••..•. •236 (U) A Summer in Philadelphia - an Exciting One ..•.••..• .238 (U) Buy a Computer, Now .••......•..•......•..•••.•.• .241 (U) Little Thanks for That Memory ......•....•.....•... .244 (U) Saving a Reputation through Logic ..•.•...•.•...••. .247 (U) The Army's Problem •...••...•..•...•....•.•..••.. .248 (U) Stratton 's Dream Revisited •......•..••..•.........• •251 (U) So Much for Simplicity ...•.••............••..•.••. .253 (U) Abner's Not Quite Best Friend •••...••..••.••.•••... •254 (U) Abneri by Inertia •••••.•••••••••••••••••. _••• ! • • • • • • • 255 (U) Abner's Bad Temper ..•..•..•......•..........•.... .256 (U) And Then_ Came ••••••••.••.•••••••••••••.• ! • • • • • • • • 2.57 Chapter 9 (U) Wandering into Trouble (U) A Cryptoanalytic Future ........•.•...•.•.••....... .263 (U) 11te Worst ofTimes ..•...•.••.••........•.•.•..... .263 (U) The Magic Continues •••.•.••••••••••••••••••••••.• •265 (U) At Last, the Electronic Bombe - Perhaps .•..•.•..••.• •266 (U) "tt'ithout Magic and without Many Friends ........... .268 (U) The End ofan ERA ...•..•..•.•.•...........•....•. .269 ([l) ERA. 's Treasures .•• -• ....•....... e: • • • • • • • • • • -• • • • • • • • 270 (U) SIGINT Loses Another Friend .•.....•.•........•.... .271 (U) An Old Friend's Burdens ...•.•.•......•....•.•.••.. •272 (U) A Desperate Search for "Depth" .....•.............•. .274 (U) Wanderers and Nomads and Chaos .••..••.•••....•• .276 Page viii IQD secRF:WCOMINl'11 PFL l'O UiA4 AUi 1 ~AN; wliilR ANQ Nd:b/JX1 IOP SECREll/COMINl/IREL TO tJSA; AtlS, CAl4, Ge" ANO NlUIXI (U) IfYou Can't Trust Someone from the Adams Family, Then.. . .......•..•.......•...... .278 (U) Failure upon Failure ......•...•..••••••.•....•..•• •280 Chapter 10 (U) A Matter of Faith (U) lVould Science or an Old Tactician Save the Agency? .• .285 (U) Rushing "Bits," Not Even ''Bytes," into the Agency ••.. •285 (U) Canine Guards the ~Fort .••••••.•••..••.••••••.•••.. .287 (U) EnterTom Watson and IBM •••••.•..••...•••••..•.• •289 (U) A Machine for U.ti, Perhaps •..•....•.•...•..•••.••.• .292 (U) One Big ll1achine Beats Out Many Little Ones •••••••• •293 (U) An ERA by Any Other Name Is IBM ....•..••.••••..• •295 (U) Is Half a Farmer Better Than ...•.••••••••.••••••••• •296 (U) St. Paul in Mohansic ..•....•.•.••••.•....•.••..•.. .297 (U) Bucks Talk: the Favored Sister Gets the Attention •••. •297 (U) A Data Factory ................................... .298 (U) Not a Farmer, a Nomad •....••.••..•••••••.•..••.. •299 (U) Engineering Is Not Science, at Least to the SAB. • •.... •300 (U) You Can Take Science Out of the Agency, But Can You .302 (U) Almost in Science - Would Lightning Be the 0th.er l"larvest . ................................. .305 (U) 'tt7hat Kind ofFriend Are You, Dr. Baker? •••..•.•••.• .307 (U) Dr. Baker's Half-an-Institute •••...••••...•.••.••••• •309 (U) A Harvest of Overexpectations •........•••••....•.•. •3io (U) ERA,'s and the Shop -floor Cryppies' Revenge ••••..••• •313 (U) Technology and Faith, i962 ••••••.••..••••••••••.•• •316 we SEC~ErMCQHIN+'JRl!b :rg YGA; AYG, SAN, GBR ANB NZLJH(1 Page ix 'FeP seeRE'fN88MIN'fNREL 'Fe tl9A; Atl9, eAN, 8BR ANB N!l:ffl(1 This page intentionally left blank Page x lOP &EBRE'.:RIG9MIN=FJ/REL le tl9A; Atl9, eAH, 81!Ht AMI') N!Uf)(I TOfl Sl!CftEtiteOMINTHREL Te 1:19A; Al:l8, 8AN, 0BR AN9 NibA'M4 Foreivord (U) Conventional wisdom about NSA and com- puters has it, as a retired NSA senior officer once wrote me, "In the early days, NSA and its predecessor organizations drove the computer industry. In the 1960s, we kept pace with it. We started losing ground in the '70s, and in the 'Sos we struggled to keep up with the industry." (U) True, but underlying this, in each decade the cryptologic organizations experienced a wide range of successes and failures, positives and negatives. If, as slang puts it, "they won some, lost some, and some got rained out," all of this experience is worth serious examination by students of computers, cryptanalysis, and NSA history. (U) The current volume, Dr. Colin Burke's It Wasn't All Magic: The Early Struggle to Automate Cryptanalysis, i93os-19(jos, contains a view of the first decades of computer development that is broad and deep and rich. (U) It begins in the 1930s as American and British intelligence officials confronted new cryptanalytic and cryptographic challenges, and adapted some intriguing new concepts to their analysis. It carries the story to the flexible and fast systems of the late 1950s and early 1960s. (U) Dr. Burke follows and links the development of automatic data processing from the critical conceptual work of the 1930s through the practical experiments born of national necessity in the world war to the postwar development and the previously untold story ofNSA's postwar computer development. Along the way, he has rescued from obscurity some important successes - and some important failures in cryptanalytic machinery from World War II. (U) All too often, discussions of NSA's computer development treat only the mainstream, ignoring the problems, failures, dead ends and might-havebeens, in order to concentrate on successes. In the present volume, however, key components of Dr. Burke's story and important for our knowledge are the machines which didn't work or which never had progeny, and why this was so. Just as important are Dr. Burke's cautionary tales about the influence of international and interservice rivalry on plans and procedures. Technical limitations and technical opportunities shaped much of the development of computing equipment, but the story is also replete with instances of man-made barriers and baleful bureaucratic bypaths that wielded great influence during much of this development. (U) A word about how this manuscript came to be. (U) A no less important factor than the information and analysis in this current volume is that it represents an objective view ofNSA's computer history by a writer not from NSA or one of the Service Cryptologic Elements. The author, a university professor, had no stake in either defending or besmirching decisions made fifty years ago or the organizations or people who made them. (U) The Center for Cryptologic History, between 1990 and 1999, administered a program to bring outside academics or researchers to the CCH for special projects. The CCH sponsored six Scholars in Residence in that period; of the six, the first two received security clearances for work on classified projects. The others remained uncleared and worked only with declassified materials. (U//FOUffl Dr. Colin B. Burke was the second of the two cleared scholars. lQR SEiCREil11COMINlJ!Alil.: lQ YGAi A\:18, SAN, GBR ANB Nlt:HJE1 Page xi 'f6P BEeREnVE:eMIN'fNREL 'Fe l:IBA; Atl8, eAN, 8BR ANB Nflfl)(1 (U//POUOj Dr. Burke's professional biography is given at the end of the book. Note that as a professor of both history and computer re.search techniques, Dr. Burke became one of the pioneers in the field of computer history. (U//FOUQ1 As background reading, I recommend not only Dr. Burke's own unclassified publications, available commercially, but two classified histories available from the Center for Cryptologic History: Thomas R. Johnson, American Cryptology during the Cold War (4 vols) and Michael L. Peterson, BOURBON to Black Friday. DAVID A. HATCH Director, Center for Cryptologic History Page xii "feP SE8REn\'88MIN'fllREL 'f8 l:ISA; AtlS, 8AN, 8BR ANB N:ZLHH1 TO,. S!CIU!!TliCOMINTli"!L TO t:ISA; At:IS, GAN, 11!" AN" N!WX I (U) Introduction (U) Before NS.4 Opened Its Files (U) I am one of those "outsiders" I talk about so much in the later chapters of this book. I was fortunate to be brought into the National Security Agency as one of the Center for Cryptologic History's first Scholars in Residence. I was borrowed from my university because I had spent a decade working on the history of computers at NSA's predecessors. I even had the courage to write a book about the subject. 1 (U) That monograph was on the machines, policies, and relationships that led to the U.S. Navy's cryptanalytic machine (computer) program in World War IL The book was also about the first major attempts to automate the American library. It had to be about both because the same people built bibliographic and cryptanalytic machines. (U) During World War II American cryptanalysts built some of the most sophisticated electronic machines in the world, but the need to address cryptanalytic crises blocked them from creating the general-purpose digital electronic computer. (U) Just as my book was published, I was asked to come to the National Security Agency. One purpose of my year in residence was to see if it was possible to write a complete history of computers at the Agency. The goal was a monograph that covered the entire life of NSA and its predecessors. The thought of finally being able to see the many highly classified documents that had been withheld from me more than balanced the pledge I had to give: I had to promise to refrain from publishing without the approval ofNSA's censors. (U) 11ie Stacks Were Not All They ... (U) My study covered events in the history of the machines through the 1940s, but its focus was on the period between 1930 and 1945. An important conclusion was that the relationship between the efforts of America's codebreakers and the emergence of the modern digital electronic computer was more complex than had been thought. The navy's cryptanalysts were in a push-me, pullme situation. Their work made mechanization a necessity, but the pressures of war and the refusal of the government bureaucracy to sponsor longterm research and development programs prevented the navy from becoming the inventor of the modern computer. (U) I began my residency by surveying the Agency's archive holdings and by rereading the few synthetic works that had been declassified. The comprehensiveness of the archive holdings was critical because unless enough of the correct type of documents had been saved and indexed, there was little chance to produce a history of the post-World War II era. Useful documents from the 1950s through the 1980s were of special importance because the Agency had allowed almost nothing about its operations in the last forty years to be made public. There was not even a counterpart to the informative but very unhandy collection of documents on the pre-1946 period released to the National Archives, the Special Research History series. f>etitcd Ftmn. l'JS/z;'SSSAI lli!J 11 Bated 24Fih1rm1y 1998 Bcclassif3 811. Ht (U) As I examined the collections, I was pleased to find that the materials I needed to l9P 81!6Rl!l!.'69MINlHREL l91::16A; M:16, 6itcN, 8BR itcNB NZLJIJ(1 Page 1 T-9P 8E6AElH89MINT.VAEI:: Te l:ISA; Al:IS, 8AN, eeR and tel::N)(1 revise my earlier work were plentiful and well organized. There were several contemporary studies of computer efforts up to 1945 that were technically as well as historically enlightening. (U) I was also happy to discover that although I had not been correct in all the details about the early American cryptanalytic machines, I had come quite close. I concluded that I had drawn an acceptable overview of the computer efforts of the army and navy cryptologic agencies up to 1950. (U) But I was less than content with the materials for the postwar era. I decided that it would be impossible to tell the type of story about the forty-five years since the Korean War that I had done for the earlier period, at least not within one year. (U) However, there were many documents, several oral histories, and the work of Samuel S. Snyder to provide a basis for a history of computers within NSA up to the early 1960s. (U) Ask What the Agency Didfor ... (U) Samuel Snyder was one of the founding fathers of machine cryptanalysis. He joined the army's cryptologic unit in the 1930s and remained at the Agency, becoming important to many of its computer projects of the 1940s and 1950s. His experience, his desire to document the Agency's computer history, and his commitment to the welfare ofNSA, made its administrators receptive to his requests to be allowed the time and resources during the late 196os and early 1970s to write about the Agency's computer history. (U) Mr. Snyder completed several works. They ranged from sketches of the history of the army's first electronic computer, Abner, to a survey of the general-purpose electronic computers the Agency had built or purchased. The study of the "gp" computers seemed so important and fit so well with the Agency's desire to obtain good Page 2 publicity that it was declassified and, in various open literature. 2 forms, published in the (U) Mr. Snyder's work showed the contributions of NSA to the development of computer technology and to the emergence of the American computer industry. It made it clear that NSA had been a major sponsor of technical advances. Like several other large government agencies, its computer purchases and its research and development contracts helped establish America as the world's leading computer manufacturer. He also made it clear that NSA had been at the cutting edge of computer technology and architecture. (U) I was very tempted to just deepen Sam Snyder's work on post-1950 automation, but as I went through the hundreds of record boxes at the archives and as I began to reflect on their contents, I decided that I had to do something different. I had to take an alternative view of the Agency's computer history. (U) Because of the need to guard NSA's cryptanalytic methods, Mr. Snyder could not discuss the reasons for the development of the devices he included in his published works. Just as the Agency cannot reveal its cryptologic successes without endangering them, Snyder could not give either the "why" or, in many cases, the "what" of the Agency computers to the public. The jobs the NSA computers had to perform and the decision processes that led them to be part of the famous collection of machines that once resided in the Agency's "basement" had to be left out of his studies. (U) He certainly could not discuss what were and are the most intriguing machines at NSA, its dozens of special-purpose computers, ones whose architecture embodies a cryptanalytic process. Doing more than listing their cover names would have revealed what methods the Agency was using and what targets it was attacking. Neither the NSA nor the British intelligence agencies were responsible for the initial release of information IOP SECRE lllCOMIN 11/R!t TO USA; At:IS, t"AN, el!ft ANB fti!l::N)(1 'f6P SEeRETHeeMIN'fl/REL f6 tJSA; AtiS, CAN, e!" ANO N!:tJIXI about MAGIC and ULTRA The stories were told by others. (U) The restrictions on Mr. Snyder had another influence. Because he was forced to divorce SIGINT and computer history, the machines he described seemed to have emerged as a result of an indestructible synergy between the Agency, the computer industry, and America's scientific community. (U) My knowledge of the course of technological advancement in the cryptologic arena in the 1930s and 1940s led me to doubt that the postwar era's machine history was so smooth and problem free. (U) There was something more fundamental about Mr. Snyder's approach that led me to search for alternative ways to interpret the documents that were emerging from the NSA archives and the offices of old hands at the Agency. Although Snyder's articles are invaluable, his interpretation seemed unlikely to be able to bring together the policy, the cryptanalytic, and technological histories of SIGINT computerization. (U) The emphasis in his public articles was on what the Agency's computer efforts did for others, especially the computer industry. But I had a clear sense that I could tell the story of computers only by using an approach that was the near opposite of Sam Snyder's. Focusing on NSA's role in transferring technology and supporting commercial computer development, I concluded, hid as much or more than it revealed. (U) Ask Not What the Agency Can Do for ... (U) To understand computerization at NSA, the question should not be "what did NSA do for the computer industry?" but "what was it that the industry could not or would not do for NSA?" (U) The second question leads an investigator into the many technical, institutional, and politi- cal struggles that the Agency faced as it attempted to keep up with the cryptologic capabilities of its adversaries. It helps to explain why NSA has had to build so many special computers, why it invested in several technological misadventures, and why it had to involve itself in some unusual relationships with private industry and academia. The question also helps to integrate the political history ofNSA with its drive to advance mathematical cryptanalysis. (U //FOUO) For example, using the "couldn't do" approach to Agency projects led me to an understanding ofNSA's great computer adventure, the Harvest system. It also helped uncover the Agency's reason for creating its high-powered mathematical think tank at Princeton. Both the computer and the institution were the result of much more than a desire to extend the reach of formal cryptanalysis; they were born of intense political pressures on the Agency, and they were grand compromises rather than perfect solutions to abstract problems. (U) Spotlighting what the computer industry could not do for signals intelligence also helps to integrate the Agency's computer and cryptanalytic histories. Putting them together shows that inventing and developing an effective technology for SIGINT has been difficult and, at times, agonizing. (TSf/Sif/R£L~ The "couldn't do" question also illuminates one of the most fascinating aspects of the history of computers at the National Security Agency: the drive to define and implement a computer architecture that was radically different from the classic design that is now called the von Neumann architecture. Since at least 1946, American cryptanalysts have done much more than use parallel and pipeline processing; they have sought, and came close to achieving, a unique architecture for a cryptanalytic general-purpose computer. Because the computer industry was unwilling to develop machines that served only one customer, NSA was forced to l'QP GE6RE'FN69MIN'fHREL 'f0 l:JSA; Atl9, CAN, e!R ANO N!LJ1XI Page 3 TQP 8E8RETN99MIN'fh'REL Te t:f9A; At:J9, eAN, 8Bllt and N!L/fXI design and build not only single-purpose machines but its own versions of a general-purpose computer. Its Sled and Dervish family of high-speed devices reflects the Agency's special needs and challenges. Unfortunately, creating an all-purpose special NSA computer was too much for the Agency, given its resources. Its famed Harvest computer, for example, was only a partial representation of a true "cryptanalytic" computer, partially because the computer industry could not focus on Agency needs. (U) When the "couldn't do" question is extended to American universities, it becomes easier to understand the difficult relationship between NSA and "outside" scientists and their institutions. Finding and utilizing academic talent was very difficult for the Agency. Professors did not rush to NSA before or after World War II, and they did not pursue much research that was of direct help to operational cryptanalysis. Devising ways to preserve the Agency's secrets and its independence while channeling the contributions of academics proved to be very difficult. (U) In the typical story, codebreaking successes such as MAGIC and ULTRA came about because of the quick and intense work of a handful of geniuses armed with brain power and little else. The common image is that once the likes of William F. Friedman and Alan Turing had their flashes of insight, a flow of precious information (and only valuable information) was captured, processed by a hastily constructed but ingenious machine, and then directed to decision makers. (U) Such an image of a heroic cryptanalysis is far from being true or useful. Cryptanalytic and technological victories have not come as easily as that. Even during the glorious codebreaking days of World War II, America's cryptanalysts barely kept up with their enemies. (U) There have been moments when great breakthroughs have led to critically important messages. And the penetration of some systems, such as the U-boat "E," led to a stream of immediately important information. But typical cryptanalysis was and remains a continuing struggle to discover patterns and to make sense out of mountains of raw data. (U) Inspiration and Patience (U) Something besides the relationships with the commercial computer industry and academia has to be called on to explain NSA's computer history. To show why the Agency created its own computer architecture and why it accepted some technological retrogressions calls for a bit of cryptanalytic muckraking. The craft of code and cipher breaking has to be stripped of its romanticism. (U) The stories of the successful American attack on Japan's diplomatic ciphers before World War II (PURPLE-MAGIC) and the triumph over the German ENIGMA-ULTRA are used as popular models for the way cryptanalytic work proceeds. The popular view shares much, in terms of fundamentals, with the public view of Agency computer development. Page 4 (U) Most cryptanalytic solutions have come only after years of the most tedious and disdainful work. The intellectuals that Britain gathered at Bletchley Park had to perform mind-deadening menial computing tasks hoping that all their