“F13.4b: a periodic key on 2.08 million dictionary keyword tables, key lengths 1-26, from the ciphertext alone: an exact per-column frequency screen of every table, then quadgram climbs and MIX8m on the best 50 per length”
F13.4b (issue #85). 0234 solved the sculpture's 8 table classes from the ciphertext alone; here the tables are 0109's: every distinct keyword-mixed alphabet from the dictionary's words of 4 letters or more (208,062), on the plain side, the cipher side or both, against the standard or KRYPTOS alphabet (5 pairings), added or Beaufort form: 2,080,620 tables, key lengths 1-26. A climb per table is about 45 hours per text, so two stages: (1) for every table and length, each column's best shift by English letter frequencies, which is exact per column (compiled, about a minute per text), the 50 best tables per length kept; (2) for those, 0234's coordinate-ascent climb on quadgrams from the frequency optimum and 9 random starts, the best readings rescored by MIX8m (F13.3). Statistic, copies and plants as 0234 (K4 against 100 climbed shuffles; 40 plants at lengths 5-26 on random dictionary tables, each against 10 own shuffles). Cost stated before the run: 541 texts x about 70 s, about 10.5 core-hours.
Like the last run, but the tablethe table The right-hand half of the sculpture, 26 rows of the alphabet each shifted one place further, starting from the word KRYPTOS. is built from an ordinary English word instead of KRYPTOS: any of 208,062 words, used on either side. Too many tables to try the full method on each, so first every table gets the quick hand test (for each column of the secret wordthe secret word The word or string a cipher needs besides the method., which shift gives the most common English letters), and only the 50 best tables for each word length get the full treatment. Hidden test messages, made with random dictionary tables, tell us up to what length this still finds the answer.
Words in the margin are defined on the glossary page.
1.1
The method at work
The carved letters go in at the top; the reading comes out at the bottom.
the idea
F13.4b: a repeating key on a table built from any dictionary word, solved from the carved letters alone.
the check
Every one of 2,080,620 tables screened column by column by letter counts at every length, the 50 best per length climbed; K4 against 100 shuffles.
the bound
K4 sits inside its shuffles (p 0.6634). Planted messages are found only to length 5: 1-5 closed without the known letters, 6-26 too short to decide this way.
2
Experiment lineage
What this experiment was built on, from the lineage page: 12 experiments before it, back to the root of its line. Its number is in bold.
keptreverted, in its family's colour×error or timeoutreverted under today's rulesbuilt on a parent in the same familybuilt on a parent in another family
3
Best candidate against the known letters
Laid out as the copper carries K4: OBKR ends the line that finishes K3, then three lines of 31. Left, as carved, the 24 known places in accent. Right, the candidate: a known place it gets right is an accent cell, one it misses is outlined. This is the candidate with most known letters right; none passed every gate.
in: the copper, always the same
as carved, 97 letters; the 24 known places in accent
ECDMRIPFEIMEHNLSSTTRTVDOHW?OBKR
UOXOGHULBSOLIFBBWFLRVQQPRNGKSSO
TWTQSJQSSEKZZWATJKLUDIAWINFBNYP
VTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
OBKR ends the line that finishes K3, then three lines of 31
out: the candidate
every place read; known places in accent, misses outlined
ECDMRIPFEIMEHNLSSTTRTVDOHW?ACEC
DIDEDMOZDIWELONNAMEITARHEAVASFO
RAJMCSATESTENCERTYNROAIDAPBABWI
IRTANDHUDDATEDNAASCRATNSQUSTSTO
checked: the known letters in place, and the reading turns back into the copper
4
How the score is made
The loss of the best candidate, part by part. Zero is solved.
Known letters missed
20 of 24, ten points each
200.00
English
the 73 free letters score -5.49; English text scores -3.6
1.89
Key
189 bits to write it down (key words are cheap, arbitrary letters 4.7 bits each); each 60 beyond 60 costs a point
2.16
Loss
of 240, the worst possible
204.05 85.0%
5
Walkthrough, by hand
Redo the method on a few letters with pencil and paper.
Build a table from a word, say HYDRAULIC: HYDRAULICBEFGJKMNOPQSTVWXZ.
Guess a key length and write the carved letters in that many columns.
For each column, try all 26 shifts and keep the one whose letters are most like English (many E, T, A; few Q, Z).
Do this for every word in the dictionary and keep the 50 tables that look best.
For those, improve the shifts by how well the whole text reads, and compare K4's best with what shuffles of K4 reach.
the worksheet: the first steps by hand, on squared paper
6
Worked example, generated
The same steps run by the code on the real letters.
The screen's first step on a planted message: plain alphabet HYDRAULIC-mixed, key SHADOW (6), standard cipher alphabet:
plain alphabet HYDRAULICBEFGJKMNOPQSTVWXZ
carved NHKDFRWODNIEWZHTAHVYPYOGAHESXGVPEIIGUCANZCWWKYFHYOIRYZ
key, true SHADOW
key, by letter counts per column alone: SHADOW
reading THEHOTAIRESCAPINGFROMTHECHAMBERCAUSEDTHEFLAMETOFLICKER
Every one of 2.08 million tables gets this step at every key length; the 50 best per length are then climbed.
7
Hypothesis, for a cryptanalyst
Family, key space, the constraint under test, and the clues it reads.
F1 on dictionary keyword tables (0109: exact with the known letters, nothing below 26) read without them: the hedge for A1 and A2 on the table family Sanborn's K1-K2 belong to, with C27(c) (his chart 'slightly different from the one on Kryptos') and C5 (key word/s) read as a keyword-mixed alphabet. F13.1 puts the cell at 143 bits at length 26 (122 for the key, 20.3 for the 2.08 million tables), under MIX8m's 189. The frequency screen is the hand method's first step (each column's shift by letter counts; Friedman, Military Cryptanalysis I) and is exact per column; it loses the true table when columns are short, which the plants measure. Pre-registered: one statistic per text, S = max over L of the z-score of the best MIX8m log-probability at L against the shuffles; K4 flags at p < 0.05 against 100 shuffles; a planted length is decided when at least 6 of its 8 plants are above all 10 own shuffles. The item closes for the decided lengths and is blocked above them.
8
Prediction
Written before the run; the result decides it.
Plants found 8 of 8 at length 5, most at 10, few at 15-26: the frequency screen needs columns of 7 letters or more to keep the true table among 2 million, so decided to about 10. K4 inside its shuffles (p above 0.05); its 26 readings put at most 4-5 known letters in place. F13.4b closes for the short lengths and is blocked above.
9
Method
Tables: 0109's 208,062 alphabets x 5 pairings (W/W, W/standard, standard/W, W/KRYPTOS, KRYPTOS/W) x 2 forms.
Screen (C): per text, table and length 1-26, the sum over columns of the best shift's English letter log-frequency; the 50 best tables per length kept.
Climb: for each kept table, coordinate ascent on the shifts by quadgrams from the screen's shifts and from 9 random starts; the best reading per table rescored by MIX8m (0233's weight 0.55); per length the best.
S = max over lengths of the z-score against K4's 100 shuffles; K4 ranked; each plant against its 10 own shuffles.
K4's best reading at each length submitted (key: the two alphabets, the form, the shifts); gate 4's controls are the shuffles' best readings scored as the harness scores.
10
Result and bounds
The harness's record: candidates, gates, loss, and the negative result this experiment leaves behind.
26candidates returned
0passed every gate
4/24known letters, best
85.0%lowest loss
nonechampion before
All 26 candidates, lowest loss first.
1failed at cribs4/24 known lettersloss 85.0%
ECDMRIPFEIMEHNLSSTTRTVDOHW?ACEC
DIDEDMOZDIWELONNAMEITARHEAVASFO
RAJMCSATESTENCERTYNROAIDAPBABWI
IRTANDHUDDATEDNAASCRATNSQUSTSTO
2failed at cribs3/24 known lettersloss 89.0%
ECDMRIPFEIMEHNLSSTTRTVDOHW?LCON
TOAFTEDICTEMIURAFEINOURREOROTTE
WSTDYPMTTFBCDISCHOITTIMITVECABQ
OTFHIELOSIDIMVCHDRIIHOONWUPONEY
3failed at cribs3/24 known lettersloss 89.1%
ECDMRIPFEIMEHNLSSTTRTVDOHW?AGES
ISFACEDNOARVEVELLURSHILEESPAWAR
DRONTUPAGALDIODEBARELECICRUSIEW
HORTYCLECOCTFOOTINTRONGOSUMEORH
4failed at cribs3/24 known lettersloss 89.2%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NTOB
ERMATEDCERTYPOLLBESENCAANDLANCO
ATSINBYTODIMWHUNGBEHANDRENASECR
ATTAWRSTWOHAZGALCERDCOVSAVRENUC
5failed at cribs2/24 known lettersloss 93.2%
ECDMRIPFEIMEHNLSSTTRTVDOHW?FLAO
REBCSJOBGCEPOOGLUHNTANGUCLIMBNC
RSHECLUNSUMTISACRIDRAWASAFETYPD
WDCTWOUVUIPLDURLIMBILEGMAONMESI
6failed at cribs2/24 known lettersloss 93.2%
ECDMRIPFEIMEHNLSSTTRTVDOHW?DSRS
UNGELWEPICNPFUSJEUYSVAAQRQVISCN
UEDATNACTARIZELDOHYEDFLOGGUSHXZ
FUTJILPROMCRIXTXTDFVRVMELVARTCR
7failed at cribs2/24 known lettersloss 93.2%
ECDMRIPFEIMEHNLSSTTRTVDOHW?OYFI
STAPIVATBYTRATEBECATITGAIREGYSD
SETIREKASHESOMORAFREPOWEREDAILA
RYCHICKMESCANPREGAHHHHERVETARWX
8failed at cribs2/24 known lettersloss 93.3%
ECDMRIPFEIMEHNLSSTTRTVDOHW?IGHL
CSMSPYCRSOIROESGAEZLNMFAEKPNBOI
DADFOJMBORNLVAPYCHRCTOPAUVESKTA
NDYQLEGNABDNLWYCUTOBHIKIWIRNUPE
9failed at cribs2/24 known lettersloss 93.3%
ECDMRIPFEIMEHNLSSTTRTVDOHW?COQM
CIWHEESHORTRHADHUGHWINEORLERLAC
KINTESABLIKEFANDSCISTBAIUMADUNC
USKETCPOPPHKLGRHLBGERSDVALYSIMS
10failed at cribs2/24 known lettersloss 93.3%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NGHG
PQDRCSPOPELERHPGRNDWAWOCHSVONRC
EEDORAVJETAONEBITEOFFUWKTGJEPMR
ADIDOHIORIZMCUITNCHKHOTILSTANDT
11failed at cribs2/24 known lettersloss 93.4%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NDTI
ENPERITMEDONASTELLSTHROWIYANDAG
OBAMNODTIRESHEROUTHEGRIORISENSE
RIESCLARGARTEROTTERAYOURTIUADWP
12failed at cribs2/24 known lettersloss 93.4%
ECDMRIPFEIMEHNLSSTTRTVDOHW?WSLF
ANEAREAMATIONGINTSPEEDHISAGAMBL
YVOLTYABOOKANAHUSEHABIZDNICDRUN
AMCCTUREAMMANNOVEMPLOTHASAFTIDE
13failed at cribs1/24 known lettersloss 97.3%
ECDMRIPFEIMEHNLSSTTRTVDOHW?ROEC
FCLRTISRIERMONSIAGMAHCSMSFUANGI
VEBRNAFTUSABRALINARTONKFTHETEDT
HEHUQSTADPOMAGGYNONTESXTRIGEIOF
14failed at cribs1/24 known lettersloss 97.3%
ECDMRIPFEIMEHNLSSTTRTVDOHW?MIXT
NICUINCCBCLEHMQUPOVVBTBEAFHDGAO
RPHYOUTIMERENCENAISZYHEENWONRES
DRYTEAYPEODRELLORWUEUITUMGEROSN
15failed at cribs1/24 known lettersloss 97.4%
ECDMRIPFEIMEHNLSSTTRTVDOHW?AFEH
XANATGIBLEHOLAYCHTOFSEEEDGRNLIN
EMEEPSETESTOWNTEATBOROFICSITEAL
LEHINPEOURGEUREANRUTASUNMRSCOMG
16failed at cribs1/24 known lettersloss 97.4%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NTTA
ENDCWNYLANTDOYZRWRSADAIEAHIMONT
OEDOFFIONWEBIKYONTREKOHKMBRAPAP
FDCITRESCISTHBONGYMIMNVESGFSXSU
17failed at cribs1/24 known lettersloss 97.4%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NLEB
ANXNHGAPLSNPTILLYIPBZCCDBMHESSN
OYOCSFCSSJEVVYROFEPAUTRYTMILMWD
ZOOQVIDEYHUEVXOFKUTHEAGARAJEKRB
18failed at cribs1/24 known lettersloss 97.5%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NNED
GOITINTEUSAWAYSIXHORTLITGOIONDF
MRBRAKTINSFREEINAFITSJORITYSANA
HASACEUSTBIRTHTHEPRESTCSNITATHR
19failed at cribs1/24 known lettersloss 97.6%
ECDMRIPFEIMEHNLSSTTRTVDOHW?EROF
PHRATCHHIMWORKEYHILESIAAHEFISNO
RESMIESTUERATUTEXITODELFJORNTHE
FAMSAITSMEDNEKCSEQUERETURNEREPO
20failed at cribs1/24 known lettersloss 97.8%
ECDMRIPFEIMEHNLSSTTRTVDOHW?NGST
RRSYVFRORDRJPURGHCUBYSAPOWETCUY
EZWAUQAUDTIELZFEHTOXIYWAYMLNMCK
YWELEUPTHEISDBEHHQGMSXNFWRTIHMT
21failed at cribs0/24 known lettersloss 101.5%
ECDMRIPFEIMEHNLSSTTRTVDOHW?KERJ
EONAREYSPASTENEERTHYEARTYAPOLSK
LIONSQIITERKELETERSPREDFTOFEADT
ITSTCLIGITOHINDSCHEBRAMELARVINY
22failed at cribs0/24 known lettersloss 101.6%
ECDMRIPFEIMEHNLSSTTRTVDOHW?WITO
ASMRGMELEYWEATEGIVOFFBSASGMEMAP
DLHYDPRIOSECEINSAILAONSOCWNTARD
FELIBREETOBITODIFONTTUNTREERPOI
23failed at cribs0/24 known lettersloss 101.6%
ECDMRIPFEIMEHNLSSTTRTVDOHW?COAN
OFGHATIGARATELAOCGIADEISRIGIBSO
NVINTHURIPOORVSTPHIRRVARSAREFSY
SOTAGLEDCHATHTYBEPINEDNOTPOUTHJ
24failed at cribs0/24 known lettersloss 101.6%
ECDMRIPFEIMEHNLSSTTRTVDOHW?ILIA
MBONTFAILTWENDSAISYPEISGRACOPEN
ANAORGEMEETTINSCHOICENTUGLENHIN
ELEDARNOWREPRETHSPERIUSDTWPIGBU
25failed at cribs0/24 known lettersloss 101.7%
ECDMRIPFEIMEHNLSSTTRTVDOHW?ELEA
MNERKITSCANSBITPAKNAPCCCOMPIDCH
OOPCZSCIETGOTOSPERTHNRURNDILLID
GTSTOHUEXGWECASHEIFHEYTZADTGERS
26failed at cribs0/24 known lettersloss 101.7%
ECDMRIPFEIMEHNLSSTTRTVDOHW?DEDO
HIUCSMOTSEACHIALSRITEINTERTOMME
RMSIESLINEWVIEFSCIUIBAYSENOGAGA
NHPENANTERNELCHJMDEBTBSDROHDAOC
K4 statistic 1.80 at length 5, p 0.663 against 100 shuffles; plant power L5 8/8, L10 4/8, L15 2/8, L20 0/8, L26 2/8; decided lengths [5]
What was searched, as the harness recorded it.
item
F13.4b
search reads known letters
no
alphabets
208,062
tables
2,080,620
pairings
W/W, W/STD, STD/W, W/KRY, KRY/W
forms
added, Beaufort
key lengths
1, 26
screen
per column best shift by English letter frequency, exact; best 50 tables per length
climb
coordinate ascent on quadgrams from the screen's shifts and 9 random starts; MIX8m rescoring