Research evolution: the shape of the search

The experiment lineage shows what the loop tried. This shows what happened to the research itself. Time runs left to right. The dark line is the path that survived; above it are branches that left the trunk — grey and ×-ended where they stopped; everything below ran as the mainline for a while before being replaced — including which machine the work ran on. Hover anything for what happened and why.

the trunk — what the project still does todaytried, then abandoned — the × is where it stoppedran as the mainline, then replaced — the ◯ is the moment it was retiredan insight that merged in — the arrow is where it changed the projectincident — something brokebootstrap4 areas · 6 lighting buckets · region holdoutberlin onlyviewpoint holdoutmission scoreheld-out photograph

scroll sideways → 20 July – 3 November, quiet stretches compressed

bootstrap4 areas · 6 lighting buckets · region holdoutberlin onlyviewpoint holdoutmission scoreheld-out photograph21 Jul22 Jul23 Jul24 Jul25 Jul26 Jul27 Jul28 Jul29 Jul30 Jul31 Jul1 Aug28 Aug28 Sep30 Oct31 Oct1 Nov2 Nov3 Novsix days of silencefour weeks of silencea month on hold — the laptop was neededa month of quiet — the rename to Ballparkmain — unchanged sinceSentinel-2 10 m/pxdropped in 20 minPrompt-only pivot rulesnever worked3D shadow relightingparkedRetrieval-index probeparked, not disprovenPrignitz probe · 0.113era opened, then closedGoogle tiles as a second photographlicence: smoke test onlyCross-photograph consistency lossesthree tries, none keptContinuous vantages (Prignitz's open move)0.103, revertedRegion-holdout evaluation→ held-out viewpointsMedian error as the score→ mission scoreRunPod - rented 4090 pod→ ModalModal A100 - serverlesscredits out → local M1Berlin-only scope cut→ 4 areas restoredGeometric mean→ mission scoreSame-photograph evaluation→ held-out photograph20% coverage floor→ one usable fix required4 MiB fp32 size gate→ 32 MiB fp32, ~8 MiB int8Local M1 - 1-3 h per experiment→ AWS, one GPU machine per runFable 5.1 designs→ Opus 5.5, after a bake-offpod disk fullFable hits its capCI out of disktwo writers on mainpod won't resumeloop dies unnoticedcredits exhaustedfigure agent overwrites figureslaptop sleeps mid-loopmodels quarantinednetwork dropno GPU in Frankfurt150-min cap cuts exp 28enforce it in the source, not the promptthe problem is two writers, not the platformstop guessing, profile itit is never shown the places it is tested onthe score must BE the product requirementit was starved, not refutedit knew one photograph, not the citythe code was the bottleneck, not the laptoppretrained features already cross photographsan untested idea is not a refuted oneBootstrap1 m/px orthophotosRelighting rebuiltGoes publicPivot enforced in codeOne git writerPNG decode fixberlin-slim branch0.040 - 96.5% usableback to berlin-slimheld-out photographloop resumes, locallyflight test: 91/100 arrivetraining moves to AWS0.560 - borrowed eyesnative scale: 0.2930.078 - 93% usableNOT TESTED is not refutedsix flight missionsnot merging yet

The long red lines are the story. Each is a rule that ran as the mainline and measured the wrong thing: for ten of the first twelve days the evaluation asked a question the model could not answer and the score preferred guessing to learning; from July to September it tested the model on the very photograph it had memorised; and in this era a coverage floor and a size gate, both written for an older question, each vetoed the best model of the day. The loop only fixes what its ruler measures, so every one of these had to be found by the humans looking at the results, and every result measured against them re-measured.

view on GitHub