The Venona Project: How Reused One-Time Pads Exposed Soviet Spies in America

On February 1, 1943, the U.S. Army’s Signal Intelligence Service (SIS) quietly started a small project in Arlington Hall, a former girls’ school in Arlington, Virginia. Its first analyst was Gene Grabeel, who only weeks earlier had been teaching home economics at a high school. The target was coded telegrams that America’s ally, the Soviet Union, had sent from the United States to Moscow. The Soviet system used one-time pads, numbers meant to be used once and thrown away, so in theory it could not be broken. But in the chaos early in the war, the Soviets had printed some of the same pad pages twice. The analysts who found that flaw decrypted about 3,000 messages over 37 years, until the work ended on October 1, 1980 (Wikipedia; Smithsonian Magazine). Later known as Venona, the project traced the path by which atomic bomb secrets reached Moscow, and it stood behind several of the spy cases that shook America in the early Cold War.

Watch the explainer video

Watch on YouTube

Background — Why Read an Ally’s Mail?

Soviet telegrams had been piling up for years. From 1939, the Allies collected Soviet traffic along with the German, Japanese, and other Axis messages they intercepted. After the United States entered the war, the Office of Censorship received a copy of every international cable and sent the coded ones to the SIS. At Arlington Hall, the Soviet messages simply filled one wooden file cabinet after another (Smithsonian Magazine).

The decision to read them came from Colonel Carter W. Clarke, chief of the Special Branch of the Army’s Military Intelligence Service. Clarke distrusted Stalin. If the Soviet Union made a separate peace with Germany, German forces tied down on the Eastern Front could turn west against Britain and the United States. To catch such a move in advance, Clarke judged, the United States had to read Soviet telegrams, ally or not (Wikipedia).

Grabeel came from rural Virginia. Around Christmas 1942, she met Frank Rowlett, a childhood acquaintance who was now a senior SIS official, and heard there was better work in Washington. She arrived at Arlington Hall on December 28 without knowing what the job was, and four weeks later she was assigned to the Soviet traffic (Wikipedia; Smithsonian Magazine). She and her colleagues began by sorting the messages by date and by communications circuit. Sorted that way, the Soviet traffic fell into five groups. The largest was trade traffic, much of it about Lend-Lease supplies, and there was ordinary diplomatic traffic. The other three were intelligence channels: the GRU (military intelligence), naval intelligence, and the NKVD, forerunner of the KGB (Smithsonian Magazine).

Most of the Venona team were women. With the men away at war, the Army recruited mainly women, many of them former teachers, and 90 percent of Arlington Hall’s codebreakers were women. “Most of the people working on it were women,” says Robert L. Benson, a retired National Security Agency (NSA) historian (Smithsonian Magazine).

How It Worked — Why Was It “Unbreakable”?

Soviet telegrams were wrapped in two layers. First, a code clerk looked up each word or letter in a codebook and wrote down its four-digit code group. Those digits were regrouped into fives, and then a second string of numbers, the additive or key, was added to them. The addition was done digit by digit without carrying: 7 plus 8 gives 5, not 15 (Smithsonian Magazine; Wikipedia).

The additive came from a one-time pad. Each page held about 50 groups of random numbers, and a page was never to be used again. If the additive is truly random and used only once, no amount of collected ciphertext lets an attacker recover the original. Breaking a system like this requires several messages enciphered with the same additive so they can be laid over one another, which codebreakers call depth. A one-time pad leaves no depth (Smithsonian Magazine).

The Crack — The Same Pad Used Twice

Yet there was depth where there should have been none. When Germany invaded the Soviet Union in June 1941, whole factories in Moscow were loaded onto trains and moved toward the Urals. In that chaos, the office producing the pads printed duplicate sets for a time. Wikipedia puts the duplicated key material at about 35,000 pages, produced under the pressure of the German advance on Moscow. To limit the risk, the Soviets sent the two copies of a pad to different users: one might go to the NKVD in New York, the other to the Soviet Government Purchasing Commission in Washington (Smithsonian Magazine; Wikipedia).

In October 1943, Arlington Hall began using IBM punch-card machines to compare the opening groups of thousands of trade messages. That painstaking work turned up pairs of messages that seemed to share the same pad page. Lieutenant Richard Hallock, working on the trade traffic, was the first to confirm that pages had been reused. The discovery that one-time pads had been used twice was the core achievement of Venona (Wikipedia; Smithsonian Magazine).

When two messages share the same additive, laying them over each other removes its effect and leaves only the relationship between the two strings of code groups. It was still hard. The overlaps were usually only two messages deep, while experts had long assumed that a depth of at least three was needed. And stripping the additive only exposed codebook numbers, whose meanings had to be recovered one by one. The analysts used the stock phrases of trade messages as footholds. Genevieve Grotjan Feinstein, who had made the decisive break against Japan’s PURPLE diplomatic cipher in 1940, worked out how the opening groups could reveal which pad page had been used twice (Smithsonian Magazine).

Breakthrough — Meredith Gardner and the Spell Table

Around 1946, Meredith Gardner, a linguist who had taught at the University of Akron, took up the work of recovering what the code groups meant. He broke not only words but also the spell table, which the Soviets used to spell out, letter by letter, English names missing from the codebook. On December 20, 1946, Gardner made the first break into the code. A message sent in 1944 listed prominent atomic scientists, including people on the Manhattan Project (Wikipedia; Smithsonian Magazine).

Once the spell table gave way, cover names poured out. In the Soviet traffic, President Roosevelt was KAPITAN, the War Department was ARSENAL, the State Department was BANK, and the Manhattan Project was ENORMOZ. In September 1947, Army intelligence quietly shared these results with the Federal Bureau of Investigation (FBI). Gardner began working with FBI agent Robert Lamphere, matching cover names in the messages against FBI case files (Smithsonian Magazine).

Who Passed the Secrets? — Klaus Fuchs

Among the earliest messages decrypted were ones about information from a scientist on the Manhattan Project, who appeared under the cover names CHARLES and REST. A message from Moscow to New York dated April 10, 1945, called the information from CHARLES “of great value” and asked for more technical detail. Investigations based on the Venona decrypts identified the scientist in 1949 as the British physicist Klaus Fuchs (Wikipedia). Lamphere’s memo to Gardner setting out Fuchs’s identity is dated September 26, 1949 (CIA, Venona: Soviet Espionage and the American Response).

Fuchs confessed on January 24, 1950. On the 27th, at the War Office in London, he dictated and signed a statement taken down by William Skardon of MI5 (CIA, Venona; The National Archives, UK). In it he said that at first he had meant only to tell the Soviet authorities that work on the atomic bomb was going on, but they wanted more details and he agreed. At Los Alamos, he said, he did “the worst I have done,” giving information about the principle of the design of the plutonium bomb. He last handed over information in February or March 1949 (The National Archives, UK). On March 1, 1950, he received the maximum sentence of fourteen years (Wikipedia).

The Rosenbergs — Evidence Never Shown in Court

Once Fuchs was caught, the investigation ran like a chain. Fuchs named his courier, Harry Gold; Gold named David Greenglass, a machinist at Los Alamos; and Greenglass, confessing to the FBI on June 15, 1950, implicated his brother-in-law, Julius Rosenberg (Wikipedia; CIA, Venona).

The Venona traffic mentions an agent called first ANTENNA and, from September 1944, LIBERAL. In June 1950, the FBI found that the information about this agent matched known facts about the New York engineer Julius Rosenberg (Smithsonian Magazine). Message No. 1657, sent from New York to Moscow on November 27, 1944, described LIBERAL’s wife: first name Ethel, 29 years old, married five years, a Communist Party member since 1938, who “knows about her husband’s work,” and who did not work because of delicate health (CIA, Venona: Selected Messages).

Ethel Rosenberg had no cover name. Wikipedia sums up the Venona evidence as making it clear that Julius was guilty of espionage, and showing that Ethel, while not a principal, took part in his activity and played a role in recruiting her brother, Greenglass. Yet Venona was never presented at the 1951 trial, because putting the decrypts in court would have told the Soviets about the program. The government relied heavily on Greenglass’s testimony, and he changed his story several times. The couple were executed in 1953 (Wikipedia; Smithsonian Magazine).

Alger Hiss — Who Was ALES?

Not every name Venona produced is as clear as Fuchs’s. Message No. 1822, sent from Washington to Moscow on March 30, 1945, mentions an agent called ALES and says ALES had worked with the GRU since 1935. FBI agent Lamphere concluded that ALES was “probably Alger Hiss,” a State Department official who had attended the Yalta Conference in February 1945 as a member of the U.S. delegation led by Secretary of State Edward Stettinius (Wikipedia).

In 1997, the Commission on Government Secrecy chaired by Senator Daniel Patrick Moynihan wrote that “the complicity of Alger Hiss of the State Department seems settled.” After the NSA released the original Russian texts of the Venona messages in 2005, intelligence historian John R. Schindler concluded that the Russian text of message 1822 made clear that ALES had indeed been at Yalta (Wikipedia).

Objections remain. Hiss’s lawyer, John Lowenthal, argued that ALES was a GRU agent who handled military intelligence, while the papers Hiss was accused of passing were nonmilitary State Department documents. He also argued that it would have been too risky to keep using Hiss after Whittaker Chambers broke with the Communist Party in 1938. In 2007, other researchers proposed that Wilder Foote, another diplomat at Yalta, was a better match for ALES. Eduard Mark of the Center for Air Force History and others strongly disputed these arguments (Wikipedia). In short, many researchers identify ALES as Hiss, but it is more accurate to say that Venona strongly suggests it while objections remain than to say Venona proved it.

The Leak — The Soviets Knew

Venona was top secret, but the secret did not hold for long. William Weisband, who worked as a “linguistic adviser” to the Russian section at Arlington Hall, was an NKVD agent. According to Wikipedia, he revealed the existence of the program to the Soviet Union sometime in 1945. Weisband was identified and suspended in 1950, but he was never prosecuted for espionage, in order to protect what remained of Venona’s secrecy (Smithsonian Magazine; Wikipedia).

Kim Philby of British intelligence was also briefed on Venona in Washington in 1949. Learning that a 1945 message from an agent called HOMER, traced to the British Embassy, had been decrypted, he realized it pointed to his fellow Soviet spy Donald Maclean. Maclean and Guy Burgess fled to Moscow in May 1951 (Wikipedia).

Even so, there was nothing the Soviets could do about wartime messages already in American hands. Whenever the FBI produced new leads, the analysts went back to old traffic, and between 1960 and 1980 hundreds more messages from the early 1940s were translated (Smithsonian Magazine).

Release — 37 Years of Work, Public After Half a Century

In 1978, the NSA judged that little more was likely to be found and decided to phase out the program within two years. Grabeel retired that year. In 1980, the remaining analysts boxed up the work sheets and folders, and Venona ended on October 1 (Smithsonian Magazine; Wikipedia).

In July 1995, at a ceremony at CIA Headquarters, Director of Central Intelligence John Deutch released the first group of NSA’s Venona translations. By 1996 all of them, roughly 2,900 KGB, GRU, and GRU-Naval messages, had been made public (CIA, Venona). The Moynihan Commission, which reviewed the government’s classification system, wrote that the 1995 release of the Venona intercepts gave an unprecedented glimpse into the world of codes and codebreaking and revealed new insights into controversial parts of the nation’s history (Moynihan Commission Report, Chapter 3).

What Remains — One Mistake and a Patient Team

What Venona decrypted was only part of the intercepted traffic, mostly messages from 1942 to 1945, often with gaps in the text (Wikipedia). Some cover names were never matched to real people, and some, like Hiss’s, are still argued over.

Even so, Venona makes two things clear. One is that even a cipher that is perfect in theory can fail through a single mistake by the people using it. The other is that the crack was not opened by a lone genius. It was opened by the patience of analysts who sorted and matched thousands of messages by date and number, a team made up mostly of women. Gene Grabeel was recognized by the CIA only after the 1995 release, and she never told her family about her work (Wikipedia; Smithsonian Magazine).

References

• Venona project (Wikipedia)
• Gene Grabeel (Wikipedia)
• Alger Hiss (Wikipedia)
• The Women Code Breakers Who Unmasked Soviet Spies (Smithsonian Magazine)
• Venona: Soviet Espionage and the American Response, 1939–1957 (CIA Center for the Study of Intelligence)
• Venona Part II: Selected Messages (CIA Center for the Study of Intelligence)
• Klaus Fuchs confesses (The National Archives, UK)
• Report of the Commission on Protecting and Reducing Government Secrecy, Chapter 3 (Federation of American Scientists copy)

Watch the 1-minute Short

Watch the Short on YouTube