“F14.7: plaintext style as the mask: vowel-dropped, telegraphese and abbreviated English added to the judge bench; power of the quadgram scorer, MIX8m and the blind judge against the bench's 200 hard negatives”
F14.7 (issue #93). The judge bench (reports/judge-bench.md, tools/judge_bench.py) measured clean Carter, misspellings, X stops and K1-K3's text; A6.2 (0226) added those classes to the climbs. A plaintext written to be short (Scheidt: 'you've gotta remember your process'; Sanborn's K2 'interrupted radio transmission'; 'it reads like it's disjointed', Scheidt 2015 on the plaintext) would score lower. Three new classes of 30 texts each, cut from Carter with the 24 known letters in place: vowel-dropped (every vowel removed except a word's first letter), telegraphese (articles, auxiliaries and common function words dropped, X for each full stop), abbreviated (every word of 7 or more letters cut to its first 4). Scored by the bench's quadgram measure, by MIX8m (0233) and by the blind judge, each against the bench's own 100 unigram and 100 Markov negatives (judge verdicts cached from the bench run).
Messages sent by telegraph or radio were often written short: no little words, no vowels, long words cut down. If K4the carved letters The 97 letters cut into the copper at the CIA, shown at the top of every experiment page.'s hidden message was written that way, our English detectors might not recognise it. We make test messages in those styles from a real English book and check whether each detector can tell them from English-looking nonsense.
Words in the margin are defined on the glossary page.
1.1
The method at work
The carved letters go in at the top; the reading comes out at the bottom.
the idea
F14.7: a message written short, as for a telegraph: no vowels, no little words, long words cut down.
the check
30 texts per style from Carter, against the bench's 200 English-like nonsense texts, read by three scorers.
the bound
The judge sees telegraphese and abbreviated text, not vowel-dropped; the two letter-statistic scorers see none. Vowel-dropped English would pass every scorer unseen.
2
Experiment lineage
What this experiment was built on, from the lineage page: 11 experiments before it, back to the root of its line. Its number is in bold.
keptreverted, in its family's colour×error or timeoutreverted under today's rulesbuilt on a parent in the same familybuilt on a parent in another family
3
Best candidate against the known letters
This experiment produced no candidate: the search ruled everything out before a full text was built. Its product is the bounded negative under Result and bounds below.
4
Walkthrough, by hand
Redo the method on a few letters with pencil and paper.
Take a sentence of Carter, say 'the remains of passage debris that encumbered the lower part of the doorway'.
Vowel-dropped: 'th rmns of pssg dbrs tht encmbrd th lwr prt of th drwy' (a word's first letter kept).
Telegraphese: 'remains passage debris encumbered lower part doorway X'.
Abbreviated: 'the rema of pass debris that encu the lower part of the door' (words of 7 letters or more cut to 4).
Score each with the detector and compare with the highest score any nonsense text reaches.
the worksheet: the first steps by hand, on squared paper
5
Worked example, generated
The same steps run by the code on the real letters.
Carter: THE REMAINS OF PASSAGE DEBRIS THAT ENCUMBERED THE LOWER PART OF THE DOORWAY
vowelless THRMNSOFPSSGDBRSTHTENCMBRDTHLWRPRTOFTHDRWY
telegraphese REMAINSPASSAGEDEBRISENCUMBEREDLOWERPARTDOORWAYX
abbreviated THEREMAOFPASSDEBRISTHATENCUTHELOWERPARTOFTHEDOOR
6
Hypothesis, for a cryptanalyst
Family, key space, the constraint under test, and the clues it reads.
C19 ('what's a period... special characters don't work', Scheidt 2015), C4(a) (a mask that is only a way of writing), A6 (the plaintext scores as English to our scorer) and A13 (the right decryption will look right). Every sampled close and every F13 statistic assumes the plaintext looks like prose. Pre-registered, as the bench: threshold T = the highest score any negative gets; a scorer 'sees' a class when at least 90% of it is above T. If neither the quadgram scorer, MIX8m nor the judge sees a class, the sampled closes are 'closed for ordinary English' only, and the class is written into A6.
7
Prediction
Written before the run; the result decides it.
The judge sees telegraphese and abbreviated text (90% or more) but not vowel-dropped text; MIX8m sees telegraphese only; the quadgram scorer sees none (it saw 10% of clean Carter on the bench).
8
Method
Carter's text (the bench's source), words split on spaces and punctuation; 30 windows per class from random places (seed 251), the style applied, cut to 97 letters, the 24 known letters set in place.