The long used US State Department code (Monroe Cipher) was rendered obsolete by the advent of trans-Atlantic cable in 1866. (See another article.) Not only did the limited vocabulary of the traditional code often require a single word to be represented by several code numbers, the rule posed by the cable company that all numerals must be written and charged as words led to inflation of cable cost. As soon as the flaw in the old code was recognized, the Department began to construct "a new and frugal cipher code" (Weber1993 p.149).
As long as the cable company accepts sending figures only as words, an obvious countermeasure to prevent inflation of cable cost is to use alphabetic letters rather than Arabic figures for code groups. This was exactly the strategy taken by the new code of 1867 (WE029) to replace the Monroe Cipher (WE028). A copy of the new code was sent to ministers overseas on 19 August 1867 (Weber1992 p.107).
The new code of 1867 in 148 printed pages (Weber1993 p.149) consisted of one-letter code groups (a, b, ...), two-letter code groups (aa, ab, ...), and three-letter code groups. Predictably, the one-letter code group "a" represents "the". The letter "w" is not used in the code groups because it is not used in French and thus might puzzle telegraph operators in France.
While the Monroe Cipher required "Mexico" to be transmitted as "six twenty eight six fifty one fourteen hundred four fifteen fifty one", the new code can represent it as a single code group "qvf", which results in a significantly reduced cable cost, even with the double rate imposed on messages in code. (Anyway, the cable company abandoned extra charge for messages in code (as opposed to cipher) in late 1867 (Weber1993 p.137).)
Unlike the Monroe Cipher, WE029 includes code groups for words and proper names but not those for single letters and syllables. Thus, it provides a simple substitution cipher (covering the 26 letters of the alphabet including "w") for spelling words not in the code. While this means single letters occurring in a coded message may be either a one-letter code group or a cipher letter, one might assume multiple single letters in a ciphertext to be a cipher rather than a sequence of one-letter code groups.
The new code WE029 was a two-part code (i.e., it had separate encoding and decoding tables, with the first 74 pages devoted to the encoding table) with non-alphabetically arranged plaintext words (though there were blocks within which the sequence was alphabetical). The problem is: the code groups in the decoding table were not arranged alphabetically but consisted of three separate series:
(1) The first series is: aba, abc, abd, ..., aca, acb, acc, ..., azy, azz.
(2) The second series is:
(i) baa, bab, bac, ..., baz,
(ii) caa, cab, cac, ..., caz,
...
(iii) zaa, zab, zac, ..., zaz.
(3)The third series is:
(I) baa, bba, bca, ..., bza; caa, cba, ..., cza, ..., zza;
(II) aab, abb, acb, ..., azb; bab, bbb, ..., bzb, ..., xzb;
(III) aac, abc, acc, ..., azc, bac, bbc, ..., bzc, ..., zzc;
...
(IV) aan, abn, acn, ..., azn; ..., knn,
where (1)(2)..., (i)(ii)..., and (I)(II), ... are merely included here for reference's sake. These series are occasionally broken by interspersed groups such as "ny", "nz", "lc", "ld" "teb" or sequences "rxb vyb vzb xab ..." and "abo aco ado aeo afo" (though it is possible that these are arranged in some contiguous sections on paper, which should be checked with the original).
Such complicated arrangement makes it a tedious job to look up a code group in the decoding table.
Secondly, use of the separate series results in systematically duplicated assignment of code groups. For example, the code group "baa" in (i) ("average") also appears in (I) ("hotly"), "bab" in (i) ("averse") also appears in (II) ("impotently"), "caa" in (i) ("backwardly") also appears in (I) ("hugeness"), "cab" in (i) ("backwardness") also appears in (III) ("deposition").
Moreover, in at least one instance, the encoding and decoding tables are not consistent: the code group "UX" corresponds to "reply" in the encoding table but "represent" in the decoding table (Weber p.224). (The other instance given by Weber (Weber p.259, n.69), "AID" being assigned for "forty" in the encoding table but "affable" in the decoding table, is simply due to his overlook of the double assignment of "AID" both in the series (1) ("affable") as well as in the series (3) ("forty"). Even an expert such as Weber could not find it!)
There is a more serious problem in practice. While the artificial three-letter code groups provide more patterns with the fewest number of letters than using ordinary words as code groups, a significant weakness results from its lack of redundancy. An error of a single letter makes a code group signify a completely different word.
This was exacerbated by telegraph operators' practice of occasionally merging words (Weber1992 p.108). This means a recipient cannot know whether "omsjmaybi" should be parsed as "om sj ma yb i" or "om s j may bi" or any other groupings.
One example of encoding is the following.
Another specimen is given in Weber1993, p.150. It shows one example of wrong division of code letters. HMVIL XIXJ should be actually HMVILX (cipher) IXJ (sufficiently). The vocabulary which even contained words such as "changeableness", "charming", "chimera" did not include the word "cipher" and it had to be enciphered letter by letter with the substitution table.
Predictably, the new code caused so much confusion that despatches were frequently unread until copies arrived by mail weeks later (Weber1992 p.108, Weber1993 p.149).
The problem started in the very early months of use of the code. As early as January 1868, Secretary of State William Seward complained to the Anglo-American Telegraph Company about frequent errors occurring at the trans-Atlantic line between Newfoundland and Ireland, while he thought there were no problems between Washington, D.C., and Newfoundland (Weber1993 p.150). However, being largely due to the defect of the code, problems remained after Hamilton Fish succeeded Seward.
Encoded cables between Andrew G. Curtin, ambassador to Russia, and Secretary Fish were frequently garbled. During a sensitive diplomatic case, Curtin found the code seriously deficient. (Weber p.229)
Elihu Washburne, minister to France, also experienced trouble during the critical months of the Franco-Prussian War in 1870 (Weber p.231-232). After Sedan fell and Napoleon III was captured but the French showed determination of resistance by declaring a republic on 4 September, the minister cabled "Have recognized republic. Favre [foreign minister of the new republic] expressed gratitude and profound emotion. Requests United States join other powers in inquiry for peace hope personally discretion subsequently is instructed immediately." Here, the phrase "inquiry ..." is the version decoded by Weber. The plaintext preserved in the State Department reads "intervention for peace. Hopes I may be instructed immediately." (Weber p.232, p.259 n.65)
With reference to the decoding table, the original encoded message must have been something like:
Probably, the State Department clerk considered this garbled and somehow replaced "one" (inquiry) with "une" (intervention) and regrouped the sequence "om (hope) sj (personally) ma (discretion) yb (subsequently) i (is)" into "om (hope) s (-s) j may bi", by assuming "j may bi" stands for plaintext "I may be."
John Hale, minister to Spain, also complained in the first of several cables he sent in mid-February 1869, that an encoded cable from Washington was unintelligible (Weber p.234). His successor, Daniel E. Sickles, also had trouble in decoding about the time the United States was to recognize a new republic established in Spain (Wikipedia) in February 1873. He not only had to decode a cable from Washington "based on a tolerable familiarity with the more habitual blunders arising from the transmission of this class of despatches by the morse code of characters" but more than once had to resend the decoded instructions for confirmation (Weber p.235-236).
Despite these problems, the code of 1867 was used until 1875 in more than 1000 despatches sent by cable or mail. The last use of the code of 1867 by the legations overseas was: November 1873 in London, July 1874 in Russia and Germany, November 1874 in Spain, and April 1875 in Paris. (Ministers in France, Spain, Germany, and Russia switched to a keyword cipher noted below, while no despatches from London were encoded until 1877, possibly because of the eased tension between the United States and Great Britain.) (Weber p.221-237; It should be noted that the table on p.151 of Weber1993, reproduced from Weber p.221, includes despatches encoded with other codes.)
In September 1871, when the code of 1867 was still in use with ministers abroad, Secretary Fish used a route transposition cipher with his assistants J. C. Bancroft Davis and R. S. Chews (Weber1993 p.202, p.209 n.16-18). It was similar to the system devised by Anson Stager and used by the Union during the Civil War. (By the way, Davis was the person who racked his brain to successfully decode the very first message encrypted in the 1867 code from the American minister in Turkey, which was put together by telegraph operators into one long string of connected letters (Weber1993 p.192).)
Bancroft Davis used this cipher during his mission to Geneva, where a tribunal for arbitration met in December 1871.
The tribunal was set up by the Treaty of Washington in 1871 to settle the claims (known as the Alabama Claims) of the United States against Great Britain for the damages due to the British support of the Confederates (such as building the warship Alabama) during the Civil War. (By the way, the treaty is of interest in history of the English language in that during the negotiation the British raised an objection to use of a split infinitive in the body of the treaty! (Wikipedia)) It was Davis who prepared the document to set forth the case for the United States and he served as an agent to the official US representative (Bancroft Davis (1893), Mr. Fish and the Alabama Claims, p.85, 86, Internet Archive).
However, the tribunal was almost wrecked by "indirect claims" raised by some, which the British were in no way expected to accept. Davis saved the tribunal by working with the British counterpart for an official announcement that the indirect claims were out of the consideration of the tribunal (Davis p.98-103; Haswell p.89; Report of the Agent of the United States before the Tribunal of Arbitration at Geneva (1873) (Google)). Bancroft telegraphed everything to Secretary Fish by using the cipher and obtained the President's approval (Davis p.103; Haswell p.89).
At least a telegram from Fish to Davis of 14 June 1872 is known to have used this cipher (Weber1993 p.209 n.18). It was the day before the tribunal was expected to dispose of the indirect claim issue (Davis p.98).
According to a later recollection of John H. Haswell, the exchange caused frustration on the US minister at London, Schenck, who was himself providing information and his own counsel on the arbitration in Geneva by using the code of 1867 (Weber p.226). Schenck, having no idea what was being relayed through his office, insisted on having them repeated, because he found them to be only "a jargon of unmeaning words." (Haswell p.89-90)
Fish took up again the cipher in writing to Davis in August 1873 (Weber1993 p.209 n.18).
In 1874, Bancroft Davis was appointed as minister to Germany. In December 1875, Secretary of State Fish used a polyalphabetic cipher with keyword JASMIN in his instructions to Davis (Weber p.240).
It is similar to a Vigenere cipher with a mixed alphabet. A correspondent is given a mixed alphabet and a keyword and reproduces a polyalphabetic substitution table by writing the mixed alphabet on top and its cyclically rotated versions below such that the second (third, etc.) line has the second (third, etc.) letter of the keyword below the first letter of the keyword on the top line.
To encipher the first (second, third, etc.) letter of a message, the line "1" ("2", "3", etc.) is searched for a letter that corresponds with the plaintext letter on the top line.
Just for comparison, a substitution table for the Vigenere cipher with a regular alphabet may be reproduced by writing the regular alphabet on top and its cyclically rotated versions below such that the second (third, etc.) line has the first (second, etc.) letter of the keyword below the first letter ("A") of the top line.
Use of a keyword is supposed to ensure that compromise of the cipher in one channel does not lead to vulnerability of other channels. However, the same keyword JASMIN was used by other ministers.
Correspondence with Elihu Washburne, minister to France, switched from WE029 to the JASMIN cipher in May 1875 (Weber p.238, 232). The following is a specimen of a message in cipher from a chargé d'affaires in Paris to Fish in December 1875.
In December 1875, when he used the keyword JASMIN for Davis and Washburne, Fish appears to have inadvertently used it in a cable to George Boker, minister to Russia, rather than the key Boker had sent or the key which Fish had sent earlier. Boker reported he could not read it and asked Fish to send the keyword or use SCYTHE. Boker's secretary somehow thought of using the keyword JASMIN and managed to decipher the cable (Weber p.238). (This cannot have been a shot in the dark. Possibly, the keyword JASMIN was mentioned in instructions for use of the cipher or the like.) Boker thereafter used the keyword SCYTHE.
Caleb Cushing, minister to Spain, used the keyword SPHINX (Weber p.237-238).
A weakness of a polyalphabetic cipher is that a smallest transmission error tends to corrupt the whole subsequent message (see, e.g., another article) and such a confusion was seen at least in the Spanish channel (Weber p.238).
In 1876, a completely new codebook (Red Code, to be treated in another article) for use by the Department of State was prepared, which formed the basis of several official codebooks prepared between 1876 and 1938. The "Direction" of the new codebook reflects frustrations with the codes and ciphers tried after introduction of cable telegraphy, saying Chinese, French, German, Italian, Japanese, Russian, Spanish, and Turkish telegraph operators, ignorant of English, "constantly commit vexatious and often serious mutilations of original messages", which added reason to adopt codes consisting of numbers, which Haswell believed were intelligible to those people and thus could be transmitted accurately (Weber1993 p.197-198).
Ralph E. Weber (1979), United States Diplomatic Codes and Ciphers, 1775-1938
Ralph E. Weber (1992), "America's First Encrypted Cable", Studies in Intelligence Vol. 36 No. 5 (PDF); cited as "Weber1992" here.
Ralph E. Weber (1993), "Masked Dispatches: Cryptograms and Cryptology in American History, 1775-1900" (PDF); cited as "Weber1993" here.
John H. Haswell (1912), "Secret Writing", The Century Illustrated Monthly Magazine, p.83 (Internet Archive) (The article appeared in the November 1912 issue (Weber1993 p.143, n.36).)