“Which undecided cells could any English test decide? Each blocked cell's key, in bits, against what 73 unknown letters of English can carry”
F1.5, step 1 (exact). 0120's climb failed to recover planted texts, and every cell blocked for that reason (F1.1 Beaufort 23 and 26, F1.2's eight lengths, F1.4 on free alphabets, plaintext autokey at lags 53-96, F6.1, F6.2, F8's undecided Hill settings) was blocked by the climb's failure, not by a property of the cell. Here each cell's key is counted in bits and set against the redundancy of the 73 unknown letters: the harness's own quadgram model, measured on held-out English, and Shannon's two estimates for English. A cell whose key, after the 24 known letters, is larger than what 73 letters of English can pin down is blocked for good, with a number.
Some of the cipher settings we have tested cannot be ruled in or out by the 24 known lettersthe known letters Sanborn has confirmed that positions 22 to 34 of the message read EASTNORTHEAST and positions 64 to 74 read BERLINCLOCK. alone: they fit the known letters, but so do scrambled copies of the carved textthe carved letters The 97 letters cut into the copper at the CIA, shown at the top of every experiment page.. Only the other 73 letters reading as English could tell a true setting from a false one. English is predictable, and that predictability is a fixed amount of information per letter. A setting with a very large secret (a scrambled alphabet plus a long key) has so many versions that some false version will read as English by luck, and then no test can pick the true one. Here we count, for each undecided setting, how big its secret is in bitsbits A way of counting how many possibilities a secret has: each bit doubles the count., and compare it with how much 73 letters of English can say. Settings whose secret is too big are closed as undecidable for good.
Words in the margin are defined on the glossary page.
1.1
The method at work
The carved letters go in at the top; the reading comes out at the bottom.
the idea
A cell the known letters cannot decide can still be decided by English, but only if its secret is small enough: English carries a fixed amount of information per letter, and a key with more open bits than that has false versions that read as well as the true one.
the check
Each cell's key counted in bits (a free alphabet 88.4, less the relabellings that change nothing; 4.70 per key letter), less the 113 bits the 24 known letters fix, against 73 letters times the bits per letter the quadgram scorer extracts (1.78, measured here) or Shannon allows (up to 3.97).
the bound
One free alphabet, every key length; two free alphabets to length 16 (14 with a move); Hill with any key to 26: decidable by quadgrams in principle. Two alphabets from 19 (15 with a move) and autokey lags 53-84 need a better model; lags 85-96 no English test can decide.
2
Best candidate against the known letters
This experiment produced no candidate: the search ruled everything out before a full text was built. Its product is the bounded negative in section 7.
3
Walkthrough, by hand
Redo the method on a few letters with pencil and paper.
Take one cell: one scrambled alphabet used on both sides, Beaufort form, a repeating key of 23 letters.
Count the alphabet: 26! orders is 88.4 bits; relabelling its places by x -> m x + t (12 x 26 = 312 ways) gives the same decryption, so 88.4 - 8.3 = 80.1 bits.
Count the key: 23 letters at 4.70 bits each, 108.1 bits. Total 188.2 bits.
The 24 known letters each fix one letter in 26, 4.70 bits each, 112.8 bits: 75.4 bits of the secret are still open.
Measure how much a letter of English tells the quadgram scorer: 4.70 bits minus its cross-entropy (about 2.9) is about 1.8 bits; 73 letters carry about 134 bits.
75.4 is less than 134, so a true answer of this kind would stand out from the false ones in principle; whether a search can find it is a separate question.
Repeat for every blocked cell; a cell whose open bits exceed even Shannon's most generous figure for English is closed as undecidable.
the worksheet: the first steps by hand, on squared paper
4
Worked example, generated
The same steps run by the code on the real letters.
cell: one unknown alphabet on both sides, Beaufort form, repeating key of 23 letters (F1.1)
alphabet log2 26! = 88.4 bits
same decryption x -> m x + t, 312 = - 8.3
key 23 x 4.70 = 108.1
total = 188.2
24 known letters 24 x 4.70 = -112.8
still open = 75.4 bits
quadgram scorer: cross-entropy 2.92 bits/letter on Carter, so 4.70 - 2.92 = 1.78 bits of evidence per letter
73 unknown letters x 1.78 = 130.3 bits
75.4 < 130.3: decidable in principle by the quadgram test (margin 54.9 bits)
plaintext autokey at lag 86 on a fixed table: 86 x 4.70 + 3 = 407.2 bits, open 294.4,
more than 73 letters carry even at Shannon's lowest entropy (290 bits): no English test can decide it
5
Hypothesis, for a cryptanalyst
Family, key space, the constraint under test, and the clues it reads.
No clue reading is tested; the experiment exists to turn the shared blocker of seven items into numbers (P6, issue #50). Unicity argument (Shannon 1949; Deavours 1977, cited by Bean, knowledge/sources/bean-2021-cryptodiagnosis-of-k4.txt, for Gromark's 48-letter unicity point): if a cell's key has H bits (uniform over a space of 2^H, after removing the symmetries that give the same decryption), about 2^(H - 24 log2 26) keys fit the 24 known letters, and a test with redundancy D bits per letter on the 73 unknown letters expects 2^(R - 73 D) false keys to read as English as well as the true one, R = H - 112.8. R < 73 D is necessary for any English test to decide the cell; it is not sufficient for a search to find the key. Three values of D: (q) the harness's quadgram model, D = log2 26 minus its cross-entropy on held-out English (Carter's book, which is not the table's source; and the K1-K3 plaintexts), measured here; (S1) Shannon 1951 (Bell System Technical Journal 30, 50-64), upper estimate 1.3 bits per character of 27-symbol text, about 1.6 bits per letter once spaces are dropped (5.5 characters per 4.5 letters), D = 3.1; (S0) his lower estimate 0.6 bits per character, about 0.73 per letter, D = 4.0: the most any English test could extract. Key bits per cell: a free alphabet log2 26! = 88.4; the affine re-labellings x -> m x + t of alphabet places (12 x 26, one alphabet; 12 x 26 x 26 for two) give the same decryption and are removed; a key letter 4.70; a choice among N options log2 N (27,852 type-1 moves, 5,054 hand-schedule partitions, 8 tables); a Hill matrix log2 |GL_n(Z26)| = log2 |GL_n(Z2)| + log2 |GL_n(Z13)|. Cells: F1.1 one free alphabet Beaufort L 23, 26; F1.2 two free alphabets L 8, 13, 16, 19, 20, 23, 24, 26; F1.4 hand schedules on one or two free alphabets, keyword 2-31; plaintext autokey lags 53-96 on the 8 standard/KRYPTOS tables; F6.1 one free alphabet with a move, L 6-26; F6.2 two free alphabets with a move, L 3-26; F8 5x5 affine Hill and 4x4 Hill plus a periodic key 1-26.
6
Prediction
Written before the run; the result decides it.
A scratch check before this was written put the quadgram model's cross-entropy on Carter at 2.86 bits per letter, so D(q) about 1.8 and 73 letters carry about 134 bits. Then: F1.1 at 23 and 26 (R about 75 and 90) lie under every bound; F1.2 lies under the quadgram bound at 8, 13 and 16 and over it from 19 up, under Shannon's at every length; plaintext autokey lags 53-96 lie over the quadgram bound at every lag, and over even Shannon's lower estimate from about lag 86; F6.1 under the quadgram bound at every length; F6.2 under it to about length 14; the 5x5 affine Hill and 4x4 Hill with any key up to 26 under it. So step 1 blocks for good only the long autokey lags, and step 2 (a climb with power) is what decides the rest. Zero candidates.
7
Method
Measure the quadgram model's cross-entropy on held-out English: Carter's book (letters only) and the K1-K3 plaintexts; also the spread over 73-letter windows.
For each blocked cell, count its key in bits, with the symmetries removed.
Subtract the 112.8 bits the 24 known letters fix: the residual R.
Compare R with 73 D for D from the quadgram model and from Shannon's two estimates.
Classify each cell: decidable by the quadgram test in principle, only by a better English model, or by none.
8
Result and bounds
The harness's record: candidates, gates, loss, and the negative result this experiment leaves behind.
0candidates returned
0passed every gate
0/24known letters, best
–lowest loss
nonechampion before
quadgram cross-entropy 2.916 bits/letter on Carter (2.944 on K1-K3), D(q) 1.78, 73 letters carry 130 bits (S1 227, S0 290); cells by decider: quadgram 29, better model 46, none 12
What was searched, as the harness recorded it.
family
bound, not a search: key bits of every blocked cell against the redundancy of 73 letters of English