Herbert O. Yardley, who caused worldwide sensation with The American Black Chamber, later wrote The Chinese Black Chamber, in which he described his days in China from 1938 to 1940, where he was invited by the Chiang Kai-shek's government to work on Japanese codes and ciphers. (Shortly his departure, he claimed his unit broke nineteen different Japanese systems (p.xxi; Kahn, The Reader of Gentlemen's Mail, p.198).) The manuscript, which he refrained from publishing during his lifetime (p.xxiv), was published in 1983, fifteen years after his death (Internet Archive).
As suggested by the subtitle "An Adventure in Espionage", the work mainly depicts his adventurous life in Chungking (重慶) rather than technical details of codebreaking. Chungking was the capital of the Chiang Kai-shek's government after the fall of Nanking in 1937 and was thus a target of Japanese air raids. It is like a spy novel set in wartime Chungking. Though we may better not take everything as historical records, Yardley surely knew how to tell a story.
One of the first he tackled was regular communication sent in kana Morse code every day at 6:00 am, noon, and 6:00 pm. The short messages were all similar, with the same length. From experience, Yardley thought they were weather reports.
Yardley noticed only ten kana of the forty-eight in the Japanese syllabary were used in these messages. He transcribed the kana messages into arbitrarily assigned Arabic figures, which was more convenient for non-Japanese speakers.
All the messages began with 027, which was considered to mean Chungking. The second three-figure group corresponded to the time of transmission: 231 for messages sent at 6:00 am, 248 for noon, and 267 for 6:00 pm. The third group was 495 in all except 401 in the latest message sent at noon that day. Yardley noticed that they had had a light rain for several days, but at noon that day, the sky cleared. This third group must mean the weather at the time. This report of "weather fair" may signal the Japanese planes based at Hankow to start an air raid. It was already 1:00 pm. Yardley warned the Chinese general of the expected bombing and even while he was explaining his analysis through an interpreter, the sirens started. Actually, Yardley's prediction was a "wild guess", but this success made a reputation for him. (p.53-55; Kahn, p.193)
Yardley continued to analyze the daily reports and comparison with the actual weather allowed him to reconstruct the code. The narrative goes on to locating the spy sending these meteorological reports in a shack and sending false reports after the spy was captured (p.80-86).
Codebreaking of one particular cipher is described in some detail (p.111-138).
Yardley's unit broke code used by some generals and in a few cases revealed treachery, resulting in purges (p.111)
But the code used by an English-speaking Chinese artillery officer called One-Armed Bandit in communication with Shanghai resisted their attacks. He had been suspected to be the chief of local traitors. The message consisted of one five-letter group followed by four-digit groups. Analysis showed the key changed each day. They might have been superenciphered after encoding with the standard codebook (see another article) for Chinese characters.
The clue came from the first five-letter group in a late June night. The few instances given for demonstration in the book, with the date of transmission, are as follows:
The pattern EWWEE sent on 11 June catches one's eye. The last EE may stand for 11, the date, meaning E=1, and the first three letters may be the message number. If so, GDDUG sent on 10 June may indicate U=1 and G=0. That is,
If the first three digits form the message number, GDD and EWW must be 099 and 100.
After an interruption for evacuation (because of a close hit of a bomb), Yardley wrote assignment of numbers to the letters:
It should be remembered that the key changed daily. So, "t" may mean "0" on one day but "3" on another. The assignment need to be considered for each day separately. Rewriting the assignment for each day yields the following:
In the middle of writing these, Yardley noticed letter sequences such as "th", "be", etc. are digraphs frequent in English. The lines for the 12th, 13th, and 14th clearly seemed to be English words: her, light, grain/groin. From experience, Yardley considered "The cipher is based on the first lines of different pages of an English text." If the book could be identified, it might allow reading the intercepts. But how?
At this juncture, in a dramatic coincidence, Yardley saw a woman, Shiu Chen, whom he called Fidelity in English translation of the name. He had befriended with her earlier, but she was a concubine of Wang Ching-wei (汪兆銘). The anti-communist, pro-Japanese Wang Ching-wei had parted with Chiang Kai-shek and Fidelity had left Chungking with him for Hanoi. (Wang left in December 1938 (Wikipedia in Japanese). In a chapter for January 1939, Yardley says he sensed the plan from conversation with Fidelity and alerted his superior (p.36).) Now she returned despite the fact that she was known as a concubine of a traitor. Yardley had reason to believe that she actually served (p.109, 124) Yardley's superior, General Tai Li (戴笠) (p.xv), known as "the Generalissimo's Number One" or "the Hatchet Man". So, when they were invited to the Bandit's house (they were on socializing terms (p.110)), Yardley asked her to search his library for a book that has "her", "light", and "grain or groin" as the first word of consecutive pages. Fidelity found the book: Pearl Buck's Good Earth.
Yardley borrowed a copy of the book from his interpreter's professor (without disclosing the purpose) and worked out the enciphering scheme. The key is found on the page numbered "month + day + 10." For example, the key for 1 April should be on page 4+1+10=15. (From this, the word "her" for 12 June should be on page 28, but Yardley has Fidelity say it was on page 17 (p.129). He may have mistaken in reproducing the conversation.)
This way, according to my understanding, the recipient used knowledge of the date to find the relevant page in Good Earth, which allows revealing the message number and the date. (Usually, the first group is used as an indicator to find the page according to a preconcerted rule. Inclusion of the date in the letter group seems to be only redundant.)
Yardley does not explain how to use the text of the page to decipher the four-digit figures. There can be many ways to do this, and Yardley merely explains one common way as an example (p.123): the first letters of the first line of the page are numbered according to the alphabetical order. For example,
(Of the first ten letters, "a" is the first and thus is numbered "0"; "d", the next, is given "1"; and so on. In this example, "w", occurring twice, is numbered twice as "8" and "9".)
This is used as an enciphering table for the four-digit codes from the telegraph code:
Yardley distributed among his students the nearly one hundred intercepts from the Bandit's communication as well as copies of the standard codebook of Chinese characters (gathered from telegraph offices). The progress of the deciphering interrupted with "exclamations of surprise in half a dozen dialects" is the climax of the book (p.134).
The deciphered messages revealed that the Bandit indeed cooperated with the traitor Wang Ching-wei and revealed many names.
The findings were reported to General Tai Li next morning. Then ensued inevitable purges and an unexpected ending reminiscent of movies from the 1950s.