Telegraph Regulations and Telegraph Codes

Since commercial use of electric telegraph started around 1838-1843 in England (Wikipedia, Friedman p.6) and in 1844 in America (by Morse) (Friedman p.6, Wikipedia), two major concerns were cost and privacy. The need was met by commercial codes or private codebooks for replacing words, phrases, and even complete sentences by code groups (i.e., code words or code numbers). (Even if the codebooks were published, they provided privacy from the casual eyes of the telegraph operators.) A desire to further reduce the telegram cost gave rise to various kinds of codes, which took advantage of loopholes in the regulations or were often against the regulations. Such irregular practice tended to be so widespread that they were officially accepted in the end.

The present article describes development of telegraph regulation and telegraph codes. For specific codebooks mentioned, see anothre article.

Table of Contents

Codes before Telegraphy

Code vs. Cipher in Telegraphy

Numerical Code Groups vs. Numerals Encoded with Code Words

Eligible Code Words

Selection of Safe Words

From Dictionary Words to Artificial Words

Five-Letter Code Words

Cases of Diplomatic Codes

References

Codes before Telegraphy

Predecessors

Even before the telegraphic age, there were code systems. Naval signal codes any better than primitive ones began to appear from 1790 (Wikipedia). Codes representing words and phrases with angles of bars were developed for optical telegraphy invented in France in 1792 (Wikipedia; Friedman p.6).

The very use of the word "code", which originally meant a collection of rules, to refer to a system of signals or words used to represent other words in communication developed from such usage. The earliest example in The Oxford English Dictionary in this sense is "a code of signals for the army" in 1808. (Titles of early telegraph codebooks included "vocabulary", "dictionary", "cipher/cypher", etc. The term "code" started to appear in titles around 1868-1870.)

With such a background, early telegraph codes mentioned use for signalling or for mail or post cards in their titles or direction for the user.

Telegraph Codes vs. Earlier Diplomatic Codes

Of course, codes had also been used by diplomats and military commanders.

Compared with such traditional diplomatic codes, typical telegraph codes had a much larger vocabulary. The vocabulary included not only words but also phrases, occasionally made of many words. These features were for the purpose of maximizing cost reduction by decreasing the number of words in the encoded message.

On the other hand, the huge vocabulary made impracticable the compilation work of a two-part code (i.e., one with separate tables for encoding and decoding), which gave way to a simple but enormous one-part code (i.e., the same alphabetically arranged table is used both for encoding and decoding). In order to secure privacy, many commercial codes proposed some kind of manipulation of encoded message (superencipherment) such as addition of a prearranged number to a code number.

Code vs. Cipher in Telegraphy

In modern terminology, code is a system for representing words, phrases, etc. with code groups, which may be numbers (code numbers) or common (or not so common or even artificial) words (code words), while cipher manipulates individual letters in the plaintext (either by substitution or transposition).

This distinction is relatively new and the term cipher was used to cover both modes of secret writing until the early 20th century (and still is even today in non-technical context). The US State Department code issued in 1918 (known as Gray Code or Gray Cipher) was still titled The Cipher of the Department of State. In May 1919, the State Department issued Department of State Code A1 and its decoding version Department of State De Code A1 (Weber p.249). Even an expression "cipher code" was commonly used.

The word "cipher" was used by some codebook compilers to mean "a code word" ("code cipher" was also used (Cotton Telegraph Code (1878), p.vii; Lieber's Standard (1896), p.xii)) or by others to mean "a number" assigned to a code word (Ager's Telegram Code (1880) p.iv).

In telegraphic regulations, the terms code and cipher were used with a different distinction. Generally, a code group consisting of an arbitrary sequence of figures or letters was treated as "cipher" in word counting for charging. However, the exact definition of what was eligible as a code "word" and what should be regarded as an arbitrary sequence of letters was not obvious. The series of amendments of the international regulations with regard to the definition of code suggests interaction between the regulations and the practice (and abuse) of the users.

In 1932, any requirement for being a code "word" would be completely relinquished. After this official admission that code words and cipher words could not be distinguished in form, the only difference left between code and cipher was in the way plaintext was converted, that is, the distinction in the modern terminology of these terms.

Word "Code" in Official Translations of International Telegraph Regulations

"Code" or "code language" corresponds to langage convenu in French but the English translation of the International Telegraph Regulations of 1872 and 1885 used a literal translation "preconcerted language". The translation of the 1903 Regulation adopted the word "code" for langage convenu.

In the 1885 edition, the English word "code" was used in the following contexts:

(i) as an equivalent of vocabulaire (i.e., codebook) (VIII. 4 "Proper names cannot be employed in the composition of codes." (However, the next paragraph is translated with "vocabulary": "The original sending office can demand the production of the vocabulary".);

(ii) as an equivalent of indicatif (XXXVI. 1. "the code of the office called"; also XXXVI. 2, XXXVII. 1);

(iii) in the context "in the official codes" (XXXVII. 5. en signes conventionnels), meaning either "in Morse code" or "in Hughes code";

(iv) in "universal commercial code" (code commercial universel) (LXII. 2. "[The semaphoric telegrams] should be written ... or in signals of the universal commercial code. In the latter case they are considered as cipher telegrams."; also LXIII. 1, 2, LXIII 1).

The 1903 translation, which adopted "code language" for langage convenu, reserved the word "preconcerted" as an equivalent for conventionnelle in Article XIII.10 ("The address may be written in a preconcerted or abbreviated form." / L'adresse peut être écrite sous une forme conventionnelle ou abrégée), which was adapted from the previous version "... forme convenue ...."

Numerical Code Groups vs. Numerals Encoded with Code Words

From Figures to Dictionary Words

Early telegraph codes used figures as code groups, as had diplomatic codes and maritime signalling codes before the telegraphic age. Figures were suited for manipulating for secrecy (superencipherment). The very first two codebooks published in 1845 proposed to keep secrecy of the message by addition/subtraction of a prearranged number to/from code numbers (see another article). Many codebooks before the mid 1870s featured similar schemes.

Presently, figures gave way to ordinary words. Friedman cites Buell's Mercantile Cypher for Condensing Telegrams (1860, Buffalo) as the first of such codebooks he could find in the US Library of Congress but he says such codes might have been used as early as 1850. Indeed, codebooks of John Wills (1846, Baltimore) and Alexander Jones (1848, New York) used code words.

In this regard, it is interesting to observe the International Telegraph Convention first mentioned the possibility of ordinary words used as code words only in the 1872 revision (Art. 9; see the next section). It appears that such usage was not recognized in the first 1865 Convention. (It may be remembered that America did not join the international regulation until after World War II (ITU). Britain joined from the second convention of 1868.)

Bolton's Patent Code (1868, London) used numerical code groups but Bolton's Telegraph Code (1871, London) provided for word code and letter code in addition to such number code.

On the other hand, one codebook by Martin K. Thompson (1867) (see below) already devised a system of code words such as CABUHUC or FEDIXIB, where each syllable had a meaning.

Considering that choice of words was already discussed in 1869 ("Construction of Cyphers, Necessity of Choosing Cypher Words Dissimilar in Telegraphic Orthography" quoted below), use of dictionary words as code words was common at least by the late 1860s.

Merits of Dictionary Words

The inauguration of trans-Atlantic cable in July 1866 may have promoted use of words by requiring all numerals be written and charged as words, which inflated cable cost of sending code numbers and caused replacement of a long-standing diplomatic code of the US Department of State (see another article).

Further, since code numbers had been assigned consecutively (unlike the Japanese naval code JN-25 during World War II, which used only multiples of three), an error in a single digit turned a code number into another with a completely different meaning. This problem was far more serious for figures than for ordinary words, for which the human eye could correct some errors.

Figure Codes (Encoding Numerals with Code Words)

Liability of figures to transmission errors was recognized at least by the early 1870s (e.g., Watts's Telegraphic Cypher (1872) p.iii, Meyer's General Telegraph Code (1874)) and led to what was called "figure codes". Contrary to representing words with figures, a figure code was for representing figures with code words.

Meyer's case may be to the point (see another article). In his 1871 Cotton Telegraph Code, he proposed to send a check number, while his codebooks from 1874 on observed that figures were "very liable to be telegraphed incorrectly" and recommended sending a checking number by a code word corresponding to the number. He was once persuaded to publish a Numeral Code (1874), while admitting its limited use, but in his Globe Commercial Telegraph Code (1882), he reverted to his original opinion with a fresh bitterness: "all figure and other complicated combinations are dangerous in the extreme".

Meyer adopted a new approach in The Appendix Telegraph Code (1880), in which code words (called "ciphers") were numbered. The numbers were not supposed to be sent as code numbers. Instead, the preface emphasised that it was to "provide ciphers for numerals up to about 5,000."

The idea was anticipated by Ager's Shipping Telegram Code (1877) and followed by the fourth edition (1881) of the ABC Code, among others. Earlier on, The Three Letter Code (1876) by E. Erskine Scott and even Telegraphic Congressional Reporter (1847) by John Wills had implemented similar ideas. Whitelaw's 25,000 English Words (1879) simply listed code words and numbers, without any plaintext phrases to go with them. Macgregor's Variation Table (1881) mentions using any ordinary code such as Whitelaw's, Ager's, ABC, Stracker's (called "Number Codes" (p.vi)) to convert a number to a code word. Now it was recognized that the numbering of any codebook could be used to express numbers with code words, even it was not intended as such.

Further Disadvantages of Numerical Code Groups

Ager (1877) explained he employed the above scheme because "it is anticipated that a future 'Telegraph Conference' may reduce the facilities afforded for employing 'figures' in Telegrams." Indeed, from 1879 to 1896, the international regulations stipulated that a sequence of figures should be counted as one word per three figures for extra-European communications, while the rate was one word per five figures within Europe. This substantially precluded use of code numbers for trans-Atlantic cables because the 1000 combinations afforded by three figures were far short of the necessary vocabulary.

Continental Codes

Kahn (p.839) remarks that the Continental codebooks he mentions (Sittler (1868), Bazeries, de Viaris, and Baravelli (1894, 1896)) were mostly numerical and, being easier for superencipherment, seem to aim quite as much at secrecy as at economy, in contrast to American public codes with dictionary words.

Indeed, "La cryptographie militaire" (1883) (online) (Note 19) mentions a scheme of replacing the most important words with other words diverted from their usual meaning but the examples given are only those classic code names as representing Queen Hortense by "M. Antoine", Louis-Napoleon by "Mme Charles", and England by "Mme Lirson" and no reference is made to codebooks that use dictionary words as code words. Code words were used in Code télégraphique français pour réduire le coût des télégrammes et en assurer le secret (1888) by Armand Coste as well as Dictionnaire pour la correspondance télégraphique secrète et économique (1893) by Mignon but Coste mentions the English and the Americans were more advanced in this field and Mignon uses rather a cumbersome system as was seen in Slater's Code (1870) in England.

Eligible Code Words

Code Words vs. Ordinary Words

The first International Telegraph Convention made in Paris in 1865 stipulated that a message written "in figures or in secret letters" was to be charged for at the rate of five figures/letters per one word (Article 34). The next Convention of 1868 applied the five-letters-per-word rule to not only messages in figures or in secret letters but also "those considered as secret messages" (Article 37). Charging based on such a presumed status, however, had to be abandoned in the next 1872 Convention, which at least provided sort of definitions for code language (preconcerted language) in a new Article 9 as follows.

1872 Convention
Art. 9
Messages in plain language must offer an intelligible sense in any one of the languages used in the territories of the contracting States, or Latin....
The following messages are considered as secret :--
1st. Those which are composed of figures or of secret letters.
2nd. Those which contain any series or groups of figures or of letters, the commercial meaning of which may not be understood by the original sending station.
3rd. Messages containing passages written in words of preconcerted meaning, incomprehensible to the departments, or containing words not forming part of the languages mentioned in the first paragraph of the present article.

Code words could not be definitely distinguished from plaintext words. While the Regulation stipulates "may not be understood", is it not often the case that plaintext messages "may not be understood"? Thus, no distinction could be made between telegram fees for plaintext words and code words (as long as they were words).

1872 Convention
Art. 36.
The maximum length of a word is fixed at seven syllables; any excess is counted as a word.
...
Numbers expressed in figures are counted at the rate of five figures to a word, plus one word for any excess. The same rule applies in counting groups of letters . ...

Art. 37
In messages containing secret language the address, signature, and portions of contents expressed in ordinary language, or in words of preconcerted meaning, are counted in conformity with the preceding article.
The portions of contents composed either of figures, or secret letters, or of a language inadmissible under Article 9, are counted as follows:
All the characters, figures, letters or signs are added together. The total is divided by five, and the quotient (plus one for the remainder if there be any) gives the number of words to be charged for. The signs which separate the groups are counted unless the sender expressly requests that they be not transmitted.

This means, a usual word used in code language was counted as one word as long as it did not exceed seven syllables, while code numbers and code words made of an arbitrary sequence of letters (i.e., ciphers according to the forthcoming definition in the telegraph regulations) were counted at the rate of five letters to one word.

As long as code language could not be unambiguously defined, special regulations against code language was doomed to fail.

Webster as a Reference

There was a controversy about what should be eligible as code "words" (as opposed to cipher, which should be counted at a rate of five letters to a word) in public newspapers in England during March 1874 and, as a result, it was officially announced (presumably by the British telegraph administration) as follows (The ABC Code (1874), p.iii).

Code messages composed of words which can be found in ordinary dictionaries or gazetteers, but which have no consecutive meaning, are accepted at the same rate as ordinary messages. Webster's dictionary to be the standard for English words; all proper words, names, &c., which are commonly written and recognised as one word, are to be accepted and charged as one, when so written by the sender.

As a result of this, The ABC Code, just published in the year before, was revised such that "the whole of the Code words used in this work are to be found in Webster's dictionary (all compound words being excluded). Those in the first part are ordinary English words, and those in the second part are proper names, and will be found in the appendix to the said dictionary." Other codebooks also claimed authenticity of their code words, often referring to specific dictionaries.

While many dictionaries were published under the name of "Webster", "Webster's dictionary" at this time must have referred to American Dictionary of the English Language of 1864, containing 114,000 entries, which acquired an international fame and became the first to be known as "the Unabridged" (Wikipedia).

However, "the Unabridged", containing compound words and detailed definitions and other information, was too much for use as a source of code words. Some code compilers rather used pronouncing dictionaries such as Nuttall's Standard Pronouncing Dictionary of the English Language (1869) and Walker's (probably Walker's Pronouncing Dictionary of the English Language by P. Austin Nuttall (1873) (Google), containing an appendix of place names).

Maximum Word Length

The 1875 conference in St. Petersburg, which simplified the International Telegraph Convention and moved specific details to the Regulation, made one significant change with respect to telegraph codes: the maximum length of a word was specified in Article XXI of the Regulation as 15 characters within Europe and 10 characters for extra-European messages. This was substantially a reduction from the previous "seven syllables" of Article 36 of the previous convention. (The distinction between the European and extra-European regimes would be abolished in the 1896 Regulation made at Budapest such that the rate of one word per "15 characters" was applied for plaintext messages in both regimes (XX 3).)

Attempts to Constrain Code Words

The 1879 Convention at London provided more detailed specifications for code and cipher.

1879 Regulations
VIII.
1. By code language is understood the employment of words which, while presenting in the case of each an intrinsic sense, do not form phrases intelligible to the Administrations engaged in transmission.
2. These words are extracted from vocabularies admitted for international messages in code language, but the composition of the vocabularies differs according to whether it pertains the European regime or the extra-European regime.
3. In the European regime, telegrams in code language must contain only words belonging to [one of the languages designated by the contracting states]. Each telegram must contain words taken only from one and the same language.
4. In the extra-European regime, telegrams in code language can contain only words belonging to the English, French, German, Italian, Dutch, Portuguese, Spanish and Latin languages. Each telegram may contain words taken from all the above-mentioned languages.
5. Proper names cannot be employed in the composition of codes. They are only admitted in telegrams in code language, with their signification in plain language.
6. The original sending office can demand the production of the vocabulary, for the purpose of controlling the execution of the preceding regulations, and verifying the authenticity of the words employed.

IX.
1. The following are considered as telegrams in cipher:--
a. Those which are composed of figures or in secret letters .
b. Those which contain, either any series or groups of figures or letters, the meaning of which may not be understood by the original sending office, or words, names or collections of letters, not fulfilling the conditions required for plain language (Regulation VII.) or code language (Regulation VIII.).
....

Prohibition of use of proper names, explicitly admitted in England in the 1874 announcement, forced revision of codebooks. For example, the fourth edition of The ABC Code (1881) replaced the whole code words in Part II. Meyer had to re-compile his International Mercantile Telegraph Code (1875) to Commercial Telegraph Code (1880).

In the next 1885 Regulation, the rules for the language for code words were unified for both regimes and the maximum length was specified for code words. The definition of cipher was slightly changed.

1885 Regulations
VIII.
1. By code language is understood the employment of words which, while presenting in the case of each an intrinsic sense, do not form phrases intelligible to the Administrations engaged in transmission.
2. These words are extracted from vocabularies admitted for international messages in code language.
3. Telegrams in code language must contain only words of 10 characters at most, belonging to the English, French, German, Italian, Dutch, Portuguese, Spanish and Latin languages. Each telegram may contain words taken from all the above-mentioned languages.
4. Proper names cannot be employed in the composition of codes. They are only admitted in telegrams in code language, with their signification in plain language.
5. The original sending office can demand the production of the vocabulary, for the purpose of controlling the execution of the preceding regulations, and verifying the authenticity of the words employed.

IX.
1. The following are considered as telegrams in cipher :--
a. Those which are composed of figures or of letters having a secret meaning.
b. Those which contain, either any series or groups of figures or letters, the meaning of which may not be understood by the original sending office, or words, names or collections of letters, not fulfilling the conditions required for plain language (Regulation VII.) or code language (Regulation VIII.).

This 1885 revision stipulated that in mixed telegrams, plaintext words and code words were to be counted according to their respective rules (XXIX) but the distinction was abandoned in the next 1890 Regulation (XX. 2). It was brought home afresh that there was no definite way to tell whether individual words were plaintext words or code words, though one might tell whether a whole message belonged to plaintext or code language.

Selection of Safe Words

Problem of Orthographically/Telegraphically Similar Words

All words found in a dictionary could not be used indiscriminately for code language. Although a few errors in ordinary words (as opposed to random sequences of letters) may be corrected by a human reader, there are many cases in which an error in a single letter changes a code word to a completely different word: jeering/peering, morning/moaning, iconical/ironical. Even if one limits to cases which may easily occur in Morse communication, examples abound: marrow/narrow, denial/menial, batted/baited, amending/attending, baneful/dutiful (the last example being caused by erroneous grouping of dots and bars of Morse code). Further errors were caused by the telegraphy system at the time, which required one operator to tell words or letters aloud to another operator: accept/except, counsel/council, serial/cereal. Some of these errors may be corrected by a human reader in plaintext messages but it is not the case when these words are used as code words out of any context. There were so many instances of these errors that there was a book containing some 70,000 examples of errors in code words compiled from actual telegrams: Guide to the Correction of Errors in Code Telegrams (4th Ed., London, 1890). These errors are discussed in detail in Friedman pp. 16-20.

As early as 1869, an article "Construction of Cyphers, Necessity of Choosing Cypher Words Dissimilar in Telegraphic Orthography", which appeared in the 15 July 1869 issue of Journal of Telegraph (Google), recognized the problem and proposed "choosing words so dissimilar in length, as well as in telegraphic construction".

In order to to avoid telegraphically similar words, code compilers took care in selecting code words from dictionaries and consulted experienced telegraphers. In the 1870s, many codebooks claimed to have carefully chosen their code words.

Two-Letter Differential

Through the efforts to avoid similar words emerged an idea of selecting code words different from each other in more than one letter -- what was later termed "two-letter differential".

A codebook published in New York in 1880 (The Merchant's Code, extended and improved by John C. Hartfield) contained 61,000 carefully selected words. While it is estimated that there are 160,000 candidate words of at most ten letters in dictionaries of the eight languages allowed, the condition imposed on code words that they must differ from each other by at least two letters reduced the number to this (Friedman p.21). In England, Whitelaw's "telegraph ciphers" (e.g., 14,000 Latin Words, with a preface dated February 1880) implemented the two-letter differential.

From Dictionary Words to Artificial Words

Abuse of Code Words

The international regulations offered a great latitude in the choice of code words, which led to abuses by code users. One telegraph manual at the time deplored "daily use" of a code word such as CHINESISKSLUTNINGSDON, which met the requirement of at most seven syllables but contained as many as 21 letters. A codebook published in New York in 1867 (Telegraph Cypher for Transmitting Telegrams relating to Foreign News, Stocks, Gold, Cotton, Financial Matters, etc., in a Commercial Form by Martin K. Thompson) used code words such as CABUHUC or FEDIXIB, where each syllable had a meaning (Friedman p.24).

Other codes combined a root word with prefixes or suffixes having prearranged meaning. In one code, a root form ALINEA, representing "Shall -- wait for -- arrival?" may be changed to ALINEAWI, representing "Shall we wait for arrival?", where the suffix WI represents "we, us, ours", FI "you, yours", etc. (Friedman p.22, 24-25). Of course, there were other code compilers who tried to provide more realistic combinations of roots and prefixes/suffixes. Several authors used roots and inflectional endings of Latin verbs (Whitelaw (1880) and Parker (1880)). Another used prefixes "re", "pre", "un" and suffixes "or", "able", which could be relatively freely combined with many verbs (Wills (1846)).

In the 1890 conference at Paris, the Belgian delegation deplored the abuse of code language: "Certain senders introduce into their telegrams expressions which ought to be charged for as cipher language. Sometimes, each of the consonants which enter into the composition of these words has a code value; the interposed vowels serve only to give these combinations the appearance of words. Other times, the word in code language is preceded or followed by a syllable also having a code meaning. Occasionally, finally, disputes are raised relative to the current usage of the words employed." (Friedman p.29) (It should be noted, however, there was no rule that a code word must be a word in current usage until the 1908 revision.)

Underlying these problems was a fact that telegraph operators could not be expected to tell between authentic words, which may belong to any one of eight languages (see 1885 Regulation VIII 3 above), and artificial, made up words.

As long as artificial words could not be precluded with an undisputed criterion, competition among telegraph companies would pressure one into accepting sooner or later, which would inevitably be followed by other companies. Thus, through many years such artificial words were in use while not allowed by regulations of either the American communication companies, the cable companies, or the European telegraph administrations. (Friedman p.25)

Acceptance in America

A decisive factor in legitimizing such artificial words was American companies' acceptance in 1893 (Friedman p.31-32).

All pronounceable groups of letters, when such groups are not combinations of dictionary words, will be counted each group as one word. When such groups are made up of improper combinations of dictionary words, each dictionary word so used will be counted as one word.
(Book of Rules of the Western Union Telegraph Co.)

Artificial words such as Amaurecis, Adbantia, Chancin, and Interavis were among the examples given, though not marked as such. Once artificial words were widespread in America, it was inevitable that people started to use them in communication with Europe and soon pressure of competition resulted in general acceptance of artificial code words.

Code Condensers

One great advantage of those artificial code words is that they allow code condensers to reduce telegraph charge. A code condenser is a procedure to pack information into pronounceable artificial words. As one example among others, given a table for associating two-digit numbers with two-letter syllables (for example, one given in the Westinghouse Code in 1902 (Internet Archive, jmcvey)), two 5-digit numbers for two code words could be put together to form a 10-digit number, which may be converted by twos to 10-letter pronounceable "word" (Friedman p.23-24). Since typical codebooks had their entries numbered serially with, e.g., five digits, this could provide means of combining two code words into a single ten-letter code word, cutting telegraph charge by half. There were also various schemes to represent various pieces of information with roots, terminals, or syllables, which were to be put together to form a code word. According to Friedman (p.31), expressing two code words with one by using a "code condenser" or "root and terminal systems" was widespread by the 1896 conference at Budapest (see below).

Attempts for Official Vocabulary

While code compilers continued to expand the border of code words, attempts were made to put a limit on the range of allowed code words. Article VIII of the 1879 Regulations made at London (quoted above) stipulates "vocabularies admitted for international messages in code language."

In the 1890 conference at Paris, although the Belgian delegation proposed establishing an official vocabulary (Friedman p.29), the finally adopted clause was an ineffective compromise. In particular, the extra-European regime, where most abuses occurred (Friedman p.30, 26), was exempted from the rule.

1890 Regulation
VIII
2. These words are extracted from vocabularies admitted for international messages in code language, or from an official vocabulary compiled by the International Telegraph Bureau. The use of the official vocabulary will become obligatory after the expiration of a delay of three years following the date of its publication. It will be optional for the messages of the extra-European regime.

The International Bureau did take on the task and published an official vocabulary in 1894 (Official Vocabulary for Telegrams in Preconcerted Language). It listed 256,740 words of five to ten letters taken from the allowed eight languages. However, this drew many oppositions in particular from code compilers and others who did not want an official vocabulary imposed upon commerce (Friedman p.30).

In the 1896 conference at Budapest, the Belgian delegation proposed to make the official vocabulary obligatory after 1 January 1908. The proposal, supported by France, Japan, and the Swiss, met strenuous opposition. Ostensibly, it was alleged that all the authentic words in every codebook must be included in order to avoid forcing into abandonment any codebook in use. However, a real reason, albeit seldom expressed, was that obligatory use of the official vocabulary would put a stop to the convenient means of combining code words or syllables into one code group. (Friedman p.30-31)

Predictably, again, the result was a compromise.

1896 Regulation
VIII
5. From a date to be fixed by a next conference all words employed in private telegrams in code language will be taken from the official vocabulary prepared by the International Bureau, duly enlarged.

Accordingly, the International Bureau took on the task of expanding the official vocabulary to address the ostensible cause of opposition. A total of 218 codes (public and private) containing over 5,750,000 words were examined, which resulted in publication in 1900-1901 of four large volumes (New Official Vocabulary for Telegrams in Preconcerted Language) containing 1,174,864 words (which was further augmented to 1,190,000 by an appendix) (Friedman p.33). (Google has searchable (but not viewable) entries (beware the indexing accuracy is rather poor): Vol.1, Vol.2, Vol.3, Vol.4.)

At the next 1903 conference at London, Belgium and Japan proposed to make the official vocabulary obligatory when the revised regulation took effect on 1 July 1904. The British delegation read a long opposition to this proposal. After explaining many deficiencies and difficulty of enforcing the use of the official vocabulary, they came to the point: there were "codes of which the words do not comply at all with the present conditions of the regulations, but which have, nevertheless, been in use for a certain time in the commercial world, without appearing to have brought any inconvenience to the telegraph service." Further, they pointed out a dilemma that non-compliant artificial words were often easier to handle than authentic foreign words. For example, a Continental telegrapher would find "DEMINABAM" or "OPORTAVERE" (from a code used in Liverpool) easier than English words such as "strength", "bowieknife", "awkwardly". Moreover, artificial code words have an advantage of allowing various combinations freely. Thus, the proposal of obligating the official vocabulary came to a sudden end. (Friedman p.34-35)

Instead of enforcing an official vocabulary to curb use of irregular code words, discussion suddenly turned to the opposite direction, i.e., admission of such artificial words.

The tremendous labour put into compiling the official vocabularies was not a total waste, though. Many code compilers in the 1890s used the Berne Official Vocabulary as a source of code words (see another article). In particular, the numbered code words of the 1894 version were used as a figure code (for converting figures into code words) by themselves. (Such use is even described in Japanese textbooks in 1910 and 1925).

Official Admission of Artificial Words

It was the Belgian delegation that led the move to admit "euphonious combinations (real or artificial words)". Concern was expressed to the term "euphonious" (meaning "easily pronounceable"), with an example of a Polish sentence "Chrzaszcz brzmi w trzcinie", which is a plaintext meaning "The maybug is boring into the reed" but hardly euphonious to many. (Friedman p.37)

Thus, the adopted text of the new regulation stipulated "which can be pronounced according to the usage of" any of the eight admissible languages.

1903 Regulation
VIII.
1. Code language is that which is composed of words not forming intelligible phrases in one or more of the languages authorised for telegraphic correspondence in plain language.
2. The words, whether genuine or artificial, must be formed of syllables capable of pronunciation according to the usage of one of the following languages: German, English, Spanish, French, Dutch, Italian, Portuguese or Latin.
3. Words in code language must not be longer than ten characters according to the Morse alphabet.
4. Combinations not fulfilling the conditions of the two preceding paragraphs are regarded as language in letters having a secret meaning and are so taxed. Nevertheless, those which may be formed by the union of two or more words in plain language contrary to the usage of the language are not admitted.

Five-Letter Code Words

5+5-Letter Code Words

The impact of the official admission of artificial code words was immediate and far-reaching. In February 1904, even before the new regulations took effect in July, there appeared a list of pronounceable "words" titled Whitelaw's Telegraph Cyphers: 400 Millions of Pronounceable Words. There were 20,000 such "words" (called "cyphers" by the author), all made of five letters: FORAB, LUFFA, LOZOJ, FREAN. Since a code word can contain ten letters, the user was supposed to put together two such words into one code word of ten letters. (The advertised "400 millions" referred to the number of such combinations.) Thus, the telegraph cost could be cut by half. Soon this "new idea" became the standard form of telegraph code (Friedman p.39).

Whitelaw (1904) merely listed code words with numbers, intended as a figure code (see above as well as another article). It was Bentley's Complete Phrase Code (1906) (see another article) that took full advantage of the new idea in a codebook to encode words and phrases and sold widely. (Before Bentley (1906), Peycke's codes implemented the idea in America. See another article.)

It should be noted that the reduction of revenue for the telegraph companies, due to this practice of putting together two code words into one, was more than compensated for by the marked increase of the total telegraph traffic because of the affordability brought about by this practice (Friedman p.51-52, 79).

Although 5+5-letter code words were a result of bowing to the abuse of code users, they soon demonstrated their advantages over the previous artificial words. First, fixed length of code words helped error correction.

Further, the two-letter differential, which was observed by Whitelaw's Cyphers and its followers (but not Peycke's), was all the more important for artificial words, which threw away any semblance to real words and thus gave no clue for correcting errors. (Friedman p.40)

Later, it was recognized that the five-letter structure allows constructing code words with two-letter differential systematically by using a table called a mutilation table, which could also be used to correct a mutilated code word. (It is variously called a permutation table, a mutilation chart, a mutilation detector, an error detector, or a (code word) construction table (Friedman p.41; Friedman and Mendelsohn p.403).) Further, a mutilation table could also ensure there are no code words that differ by transposition of adjacent letters. Compiling code words with such features with dictionary words would have required a laborious task of finding pairs such as marital/martial, coral/carol, etc. As one example, a mutilation table is found in the Universal Trade Code (1921) (Internet Archive), compiled under the supervision of Herbert O. Yardley and Charles J. Mendelsohn. This provided a more powerful means for error correction than a terminal index found in earlier codebooks (e.g., ABC6 (1920)), a table listing all the code words in the order of backward spelling, which allowed correcting errors occurring in the beginning portion of code words. (One early example of a terminal index is seen in 25,000 English Words (1879) (Google).)

New Generation of Figure Codes

While traditional codebooks for encoding words and phrases benefited from the new 5+5 code words in reduction of cost by half, figure codes (i.e., codes to express numbers with code words according to the parlance at the time) could double the range of numbers that could be represented by one code word.

The idea was implemented in Whitelaw's own Telegraph Cyphers. Pentasyllabic words representing two series of 10-figure groups from 0,000,000,000 to 9,999,999,999 [1904]. Probably, this listed two sets of 100,000 five-letter code words, which could represent numbers 00000 to 99999. By combining two such code words in a ten-letter code words, any number from 0000000000 to 9999999999 (i.e., any ten-figure number) could be represented by one word. Such a wide range could not be attained by conventional words or artificial words that look like real words.

Also published in 1904 were what appear to be similar works (judging from the titles): Beith's 10-Letter Combinations (8 Figures) (Manchester), McNicol's Nine Figure Code, or 1,100 Millions Pronounceable Words (Manchester), and The Economical Telegram Code Vocabulary of 10,000,000 inconvertible cipher words, each of ten letters, numbered consecutively from 0000000 to 9999999. With terminational order (Bombay).

Such figure codes were not only used for transmitting large numbers accurately (e.g., by code words with two-letter difference). By assigning each digit (or a group of digits) a specific meaning, various information (particularly those provided in tabular form) could be packed into one code number. Further, when some measures are provided to distinguish code words for representing figures, they could be used to represent numbered code words of some other codebooks (see Bentley's case in another article).

Challenge to Pronounceability

Once artificial words were admitted, attempts began to be made to further extend the border of allowed patterns. It was inevitable because larger and larger vocabulary was desired to further reduce telegram cost. While Whitelaw's Cyphers had only 20,000 patterns, Bentley's Code in 1906 had 32,000 groups. Soon, 50,000 or 60,000 groups were considered insufficient and after the 1908 Lisbon conference the size inflated to 100,000 or more code words. (Friedman p.50, 56, 76)

On the other hand, the number of patterns that can be used as code words was severely limited by the principle of the two-letter differential. Taking only a single pattern of consonant-vowel-consonant-vowel-consonant as an example, 288,000 groups, of which about 200,000 are easily pronounceable, are reduced to a mere 14,400, of which only about 11,000 are easily pronounceable. (Friedman p.42)

The loophole which allowed expansion of the border of such artificial code words was ambiguity of the definition of pronounceability.

Is OHMSC pronounceable? What about PHTHIPSYCH? These patterns may seem unpronounceable at first sight. But they may be said to be formed of syllables as seen in authentic English words as "ohm", "disc", "phthisis", and "psychology" (Friedman p.47, 54).

A simple notion of pronounceability may require alteration of consonants and vowels, but phonetics teaches that some consonants are syllabic (i.e., can be pronounced singly): -ble, -sm, etc. Moreover, written syllables is another matter. Considering that pronunciation of many English words is very irregular with respect to their spelling and there are even silent letters, it makes harder to judge whether a written sequence of letters is pronounceable or not. Furthermore, even a common word such as "awkwardly" may seem totally unpronounceable for an Italian telegraph clerk (Italian does not use K, W, and Y). What about "eighths"? On the other hand, "zeeeend" is a normal (and thus pronounceable) word in Dutch (a spelling with a hyphen "zee-eend" seems to be the norm today).

In the first place, the rule is that words "must be formed of syllables capable of pronunciation according to the usage of" one of the eight admitted languages. When syllables constituting a word are pronounceable according to different ones of the eight languages, the situation would be worse for the telegraph clerk to make a call. (Friedman p.43-48)

Thus, although supporters of the new regulation had cited examples similar to real words such as DEMINABAM and OPORTAVERE, what telegraph companies faced after the regulation took effect were sequences such as EYYHOGORGT, BEEUHDWEWF, and ZAQROUARSC (Friedman p.51). Even a code compiled by one of the most scrupulously conscientious code compilers, who found evidence "Eurybates" for using a syllable "ryb", "Corvus" for "vus", etc. could produce codewords such as EQGUGUXJIK and JIUPHIQHUG (Friedman p.54-55). When the telegraph clerk found some code words unpronounceable at the counter, inevitably, disputation ensued with the sender, who would "manage by means of facial contortions to pronounce words containing up to four or five consecutive consonants" (Friedman p.51). The difficulty of judgement is illustrated by actual examples handled in the Netherlands in 1925: YIDENVOHFK and LIYFKBVEVA were deemed unpronounceable because of HFK and FKBV. These may be so. However, if taking out a sequence of letters thus is allowed, authentic words might also be rejected because of containing such sequences: BEDSPREAD (DSPR), THANKSGIVING (NKSG), ANTHRAX (NTHR) (Friedman p.57).

In the 1908 conference at Lisbon, regulation of code words was much discussed but no drastic measures were adopted. The new requirement of pronounceability "according to the current usage" was not effective. The provision that codes may be submitted to and verified by the telegraph administrations was only to assure code compilers and code users that the code words listed in the code would be accepted. (By 1921, about 108 public codes and 15 private codes were examined (Friedman p.59). Commercial codes stated the fact that their vocabulary had been verified. Two examples are: Broomhall's Imperial Combination Code ("rubber edition", 1913) and Kölkenbeck's Ideal Code Condenser (1909) (listed in another article). Even an earlier codebook such as Bentley's Complete Phrase Code had an insert in its reprint that the code words had been verified.

Possibility of tightening enforcement of the regulation was forestalled by World War I. From 1914 to 1918, censorship put a stop to international commercial communication in code almost worldwide (Friedman p.59) and the telegraph conference scheduled in 1915 had to be postponed (Friedman p.58).

After the war, use of code was resumed with more vigor than ever before. The next five or six years saw publication of more codes than in the preceding twenty years (Friedman p.59). Code compilers no longer bothered to submit their code to the authority for verification because it was now widely recognized that any code word that have the faintest possibility of pronounceability would be accepted. The distinction between code and cipher in form was now gone.

In the next conference in Paris in 1925, the inevitable proposal came out: code words must be formed of a maximum of five letters, chosen at the will of the sender, without any condition (Friedman p.60).

5+5-Letter Pronounceable Code to 5-Letter Arbitrary Code

Originally, when pronounceability was stipulated in the 1903 revision, it was intended to facilitate handling and correcting telegrams. But as a result of expanding the definition of pronounceability to the extreme, this was no longer the case (Friedman p.61-62). Increasing use of automatic printing-telegraph systems (85-90% in North America) lessened the significance of pronounceability on the receiving side (Friedman p.62). Furthermore, a survey revealed transmission of code words took only 1.56 times more than transmission of plaintext words (in the mother tongue), much less than the difference in burden felt by the telegraph operators. The extra burden may be largely accounted for by the difference between the ten-letter code length and the average length of plaintext words of 6.5 letters (Friedman p.64). When the time is compared on a per-letter basis rather than per-word, transmission of code words did not take much longer than that for plaintext words.

Moreover, five-letter code groups were easier to handle for memorizing, transcribing, and keyboard operation, etc. and less prone to error than ten-letter code words.

The United States Navy and the British Navy, where cost was presumably a second-place concern, had long before discarded the practice of combining five-letter code words into ten-letter code words in their radio services (Friedman p.67).

In the public sector, the United States, not a party to the International Telegraph Convention, had done away with the combined ten-letter code words when the 1908 Lisbon conference did not repeal the seemingly unreasonable requirement of pronounceability. As of 1 December 1909, the Western Union and Postal Telegraph enforced the new rule, whereby distinction between pronounceable and unpronounceable artificial groups of letters was thrown away (Friedman p.71).

All groups of letters, when such groups are not dictionary words, and are not combinations of dictionary words, will be counted at the rate of five letters or fraction of five letters to a word.

As a result, companies tended to continue using their old codes with dictionary words, while the same companies employed five-letter codes for international communication almost without exception (Friedman p.71).

The 1925 conference only set up a committee for the study of code language, which met in Paris in October 1925 and in Cortina d'Ampezzo, Italy, in August 1926. In 1928, a conference was held in Brussels solely to discuss the matter of code language. The result was a bizarre compromise. While it accepted "code words of not more than 5 letters subject to no condition or restriction" as Category B, it also retained as Category A "code words of not more than 10 letters" provided that they conformed to rather complicated rules of including vowels.

Doubling of cost for Category B was somewhat alleviated by stipulating the rate was 2/3 of the full rate for the extra-European correspondence and 3/4 for the European.

1928 Regulation
IX.
....
Category A. Telegrams of which the text contains code words of not more than 10 letters having at least one vowel if they comprise not more than 5 letters, at least two vowels if they comprise 6,7 or 8 letters and at least three vowels if they comprise 9 or 10 letters. In words of more than 5 letters, one vowel at least must be in the first five letters and at least one vowel in the rest of the word, subject in addition in the case of words of 9 or 10 letters to the minimum of three vowels in all. The vowels are a, e, i, o, u, y. Groups formed by combining two or more words of plain language contrary to the custom of the language to which the words belong are forbidden.
Category B. Telegrams of which the text contains code words of not more than 5 letters subject to no condition or restriction as to the method of formation of the words. Figures and groups of figures are not admitted: commercial marks consisting of a mixture of figures and letters are however accepted if the sender can prove that they really are commercial marks.

Category A posed a problem for the five-letter codes because the three-vowel requirement for ten-letter code words substantially mandated inclusion of two vowels in each half-word, which was not met by most codebooks (Friedman and Mendelsohn). Presumably, users of such codebooks were obliged to send their messages under Category B, i.e., five-letters per word.

This inconvenient Category A was discarded in the next 1932 conference in Madrid, whereby the International Telegraph Union merged with the newer radiotelegraph counterpart into the International Telecommunication Union. However, the distinction between code and cipher was still retained and different rates were applied.

1932 Regulation
X.
1. Code language is composed either of artificial words, or of real words not used with the meaning normally assigned to them in the language to which they belong and consequently not forming intelligible phrases in one or more of the languages authorised for telegraph correspondence in plain language, or lastly of a mixture of real words as defined and artificial words.
2. (1) By telegrams in code language those are meant of which the text contains words belonging to this language.
(2) The code words, whether real or artificial, must not contain more than five letters; they may be formed in any way. They must not contain the accented letter é.
3. The officer who accepts a telegram in code language enters on the form the service instruction =CDE=, which is transmitted to destination at the beginning of the preamble of the telegram.
4. CDE telegrams are charged at 6/10 of the full rate in the case of the extra-European system, and at 7/10 of the full rate in the case of the European system.
5. (1) Telegrams of which the text contains words in code language and words in plain language and/or figures and groups of figures, are considered, for the purpose of charging, as belonging to code language. Nevertheless:
(a) the number of figures or groups of figures must not exceed one-half of the number of chargeable words in the text and signature; (b) for the purpose of charging, bank and similar telegrams expressed in plain language which contain a check word or check number placed at the beginning of the text (Art. 9, 2) are not considered as code telegrams.
(2) Telegrams of which the text contains words in code language and groups of figures in excess of one-half of the chargeable words in the text and signature, are considered for the purpose of charging, as telegrams in cypher language.
6. The sender of a telegram in code language or code language mixed with another language must produce the code from which the text or part of the text of the telegram has been compiled, if the office of origin or the Administration to which this office is subject requests it.

Now, code words must be at most five letters in length. Instead, any requirement for being a code word was completely relinquished. To alleviate doubling of cost by reducing the maximum length by half, the reduced rate was applied for code telegrams. A contradiction between paragraphs 4 and 5(2) was mended in the next 1938 revision such that only code telegrams containing code words not exceeding half of the chargeable words were eligible for the reduced rate.

Private Codes with Huge Vocabulary

While removal of the pronounceability requirement allowed immense expansion of the vocabulary of a code, public codes scarcely needed more than 50,000 to 100,000 they already had, beyond which more entries would mean more trouble in finding appropriate expressions among a vast list of seldom used phrases.

On the other hand, private codes for companies took advantage of the expansion to include specific code words for their products. A cash register company's code had 1,000 code words from KAJAN to KUTAZ for various shipping instructions (e.g., KUBOR: We are shipping to you, in care of your agent at Shanghai). The old code with 100,000 entries was revised by Ernest F. Peterson to expand these 1,000 shipping instructions to 200,000 by combining each with the company's 200 models of cash registers. Similarly, Peterson also expanded a bank's code from 100,000 to 400,000 entries. (Kahn p.849)

Cases of Diplomatic Codes

The United States

When trans-Atlantic cable was introduced in 1866, the US Department of State used a figure code from a pre-telegraphic age, which, contrary to telegraphic codes, increased the number of words. Due to the rules of the cable company that numbers must be written and charged as words and also due to the length of the despatch itself, the cable charge was enormous (see another article).

The first code designed for telegraphy by the US State Department employed three-letter code groups baa, bab, bac, ... but this turned out to be a disaster (see another article).

The Red Code of 1876 caught up with the commercial telegraphic codes at the time. Words and phrases were assigned a code word (dictionary word) and a number. Its successor, the Blue Code of 1899, retained this structure. (See another article.)

The US State Department did not jump to the new practice of combining two five-letter code words into ten-letter code words, brought about the 1903 revision of the international regulation. It was only in 1910 that the Green Code adopted artificial five-letter code words: babba, babca, babda, babfa, babga, .... Disregard of the two-letter differential, having been in use in commercial codes for decades, shows the low status of American cryptology at this period.

Germany

Germany used five-figure codes.

A telegram by the US ambassador forwarded a British intercept of a German message of 25 September 1916 in German five-figure code (identified as "14417", a variant of "13040") preceded by notes in 5+5-letter American code: "PEGPYHABOG LEKOFDYLOP FAGUCHECUC MISUKFOHUS ...." (Peter Freeman, "The Zimmermann Telegram Revisited", p.104). (The first five letters represent "September 25, 9 pm", a representation of date with Green Code (Weber p.247).)

Germany used superencipherment by addition or subtraction of a 3-figure key, which was usually sent at the end of the message. For a newer code "7500", first used in July 1916, superencipherment was by addition of a repeated cycle of 3-figure keys (Freeman p.114). But superencipherment did not protect the secret from Room 40, a deciphering bureau of Britain (Freeman p.119, 143).

Denmark

A telegram from Danish ambassador Brun to the Ministry of Foreign Affairs in Copenhagen dated 26 November 1916 is presented in The Danish West-Indies and Klausis Krypto Kolumne. The code was four-digit code, sent in five-digit groups.

Sale of the Danish West-Indies (now the United States Virgin Islands) had been agreed in August. The telegram was about which documents from the secret sales negotiations could be disclosed to the public.

Japan

In 1904, the Japanese Diplomatic Code for the Ministry of Foreign Affairs was composed of code words containing 8-10 letters, which resembled dictionary words. (See another article.)

In 1922, the famous telegram deciphered by Yardley at the time of Washington Naval Conference was sent in ten-letter artificial code words. (The code composed of two-letter codes and four-letter codes, regrouped into ten-letter groups for transmission.) (See another article.)

References

William F. Friedman (1928), Report on the history of the use of codes and code language, the international telegraph regulations pertaining thereto, and the bearing of this history on the Cortina report (HathiTrust), cited simply as "Friedman" herein.

William F. Friedman and Charles Mendelsohn (1932), "Notes on Code Words", The American Mathematical Monthly, Vol. 39, No. 7 (Aug.-Sep. 1932), pp. 394-409 (JSTOR)

International Telegraph Union (ITU, now International Telecommunication Union) Pages

List: ITU Conferences Collection

1865 Paris (effective on 1 January 1866): Conference; Convention and Regulations PDF

1868 Vienna (effective on 1 January 1869): Conference; Convention and Regulations PDF

1872 Rome (effective on 1 July 1872): Conference; Convention and Regulations PDF; PDF in English

1875 St. Petersburg (effective on 1 January 1876): Conference; Convention and Regulations PDF

1879 London (effective on 1 April 1880): Conference; Convention (unchanged) and Regulations PDF

1885 Berlin (effective on 1 July 1886): Conference; Convention (unchanged) and Regulations PDF; PDF in English

1890 Paris (effective on 1 July 1891): Conference; Convention (unchanged) and Regulations PDF

1896 Budapest (effective on 1 July 1897): Conference; Convention (unchanged) and Regulations PDF

1903 London (effective on 1 July 1904): Conference; Convention (unchanged) and Regulations PDF; PDF in English

1908 Lisbon (effective on 1 July 1909): Conference; Convention (unchanged) and Regulations PDF

1925 Paris (effective on 1 November 1925, except for some provisions of rate): Conference; Convention (unchanged) and Regulations PDF; PDF in English

1928 Brussels (effective on 1 October 1929): Conference; Convention (unchanged) and Regulations PDF; Modifications of the Regulations (English)

1932 Madrid (effective on 1 January 1934): Conference; International Telecommunication Convention PDF (English); Telegraph Regulations (English)

1938 Cairo (effective on 1 January 1939): Conference; Telegraph Regulations (English/French, French)


See also another article, "Nonsecret Codes: An Overview of Early Telegraph Codes"



©2013 S.Tomokiyo
First posted on 4 October 2013. Last modified on 5 March 2020.
Articles on Historical Cryptography
inserted by FC2 system