“Not all who wander are lost”
Can a UAV learn a city by heart — no GPS, no map
on board, just a $10 flight computer?
The bottom line
Yes! We found a special-purpose neural memory architecture that ‘remembers’ an exact geo-location, given an in-flight daytime picture of Berlin — even on a photograph of the city it never saw in training. No GPS, no internet, no map on board, just a 31.7 MB file of weights. It works on 93.0% of camera frames, and it took 109 experiments and $487 to find.
95/100
flights arrive within 100 m
93% of camera frames give a usable fix
0% — never 50% 100% — every frame
mission penalty 0.078 =
7% no fix + 0.8% wrong fix
1
Berlin Spree corridor mission, camera only, GPS-denied Along the Spree over a photograph the model never saw — 7.5 km in 9 minutes, 5 false fixes at 2 places, finished 47 m from the end.
true path
believed path
usable fix
abstention
false fix
× where the false fix claimed to be
off the mapped box — no imagery, no fix
Bottom camera — what the model saw
—
128 × 128 px · 1 m/px · ~100 m AGL
2
The question One frame, no map, no internet — and it has to know when it doesn’t know.
A UAV flies over a city it has trained on. Its camera takes one picture
straight down. From that picture alone: where is it?
And — how much should the aircraft trust the answer? A wrong fix
flown with confidence is worse than no fix at all. The model has to know when
it doesn't know.
what the UAV sees — one real 128 m frame. No map aboard, no internet, no GPS.→
lat 52.49995
lon 13.35294
confidence high
what it must return — the true answer for that frame, computed on board.The first baseline answered this frame 1.52 km away at confidence 0.47 — confident and wrong, the one answer a UAV must never get. The current model answers that same frame within 48.9 m .
This, on a map, is the goal. Every dot is one held-out viewpoint: ground the model trained on, framed from a position and heading it has never seen. Left is where this started. Right is where it is now. Turning red into green — without ever being confidently wrong — is the whole project.
where this started — measured mission score 1.651 · usable fixes 3% Red is a miss beyond 250 m. The baseline is confident on every frame and wrong on every frame — the worst value the metric allows, because for a UAV a confident wrong answer is worse than silence.→
where we are — measured mission score 0.078 · usable fixes 93.0% The identical test points, answered by the current model. Green is a fix inside 100 m — close enough to correct the aircraft's drift. Nothing about the test changed; only the answers did.
3
The answer Stop asking for a coordinate. Ask which tile.
INFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss 8 epochs · MixStyle is identity at export only which weights fly changed · graph, size (31.7 MB), decode and gate identical to exp 19 θ̄ = mean of visited θ NEW — weight averaging (SWA/SWAD) uniform mean, epochs 5–8, every 4 steps ~3.4k snapshots, both members A and B BN stats re-measured on 400 batches the average flies, not the last step The champion, drawn by the design agent before it was allowed to train. One camera crop enters at the left; the trunk squeezes it to a short vector; the wide fan is the choice over all 2,970 map tiles; the small head after it is the nudge inside the winning tile. Grey is the harness's frozen contract, red is what this experiment changed , and the ochre lane at the bottom is training-only scaffolding that never boards the aircraft. Click to enlarge, or see every proposal on the model designs page.
Ask a network for latitude and longitude and it fails badly: when it is
unsure it has no way to say so, and hedging toward the middle of the map is
what keeps its average error down. Our first attempt was confident on every
frame and wrong on every frame.
So we stopped asking for a coordinate and asked a multiple-choice
question instead. Berlin is cut into 2,970 tiles of 128 m and
the network points at one. Now the answer carries its own certainty: sure,
and the belief piles onto a single tile; lost, and it smears across the city
— and the UAV says nothing instead. A second output nudges the fix to
a precise point inside the chosen tile.
Then we learned it had memorised a photograph, not the city. In
July the champion scored 0.040 — on the very
aerial photograph it had trained on. Shown a second photograph of the same
streets, Berlin’s 2024 survey, it scored 1.873 :
usable on 0.0% of frames, confident and wrong on
87%, which is what a random guess over the city scores. It had
learned one file’s pixel values; adding 20 to every pixel of its own
training photograph was enough to break it. An aircraft’s camera will
never see that file.
So the ruler changed. Since 28 September every model trains on
three photographs of Berlin taken in different years and seasons, and
is scored only on a fourth , the 2024 survey, which it never sees in
any form. Each step below is one experiment the loop proposed and
kept:
exp. the change mission score usable on 2024
1 Baseline seed 1.651 3% 2 Paired-survey batches 1.583 7% 4 Exposure-and-sharpness nuisance randomisation 1.271 12% 7 Two-member committee 1.096 8% 11 One direction per tile 0.963 11% 12 Borrowed eyes 0.560 50% 13 Deeper borrowed eyes 0.435 60% 14 Cross-survey style mixing inside the borrowed eyes 0.395 66% 15 Look at Berlin at the borrowed eyes' native scale 0.293 74% 16 Better borrowed eyes 0.155 87% 19 Sharper borrowed eyes 0.085 92% 20 Fly the average, not the last step 0.078 93%
Lower score is better: 0 is a usable fix on every
frame, 1 is silence, 2 is confidently wrong everywhere. The shaded row is where the trunk stopped being learned from scratch: from experiment 12 on, the network starts from a small ImageNet-trained image model (torchvision, BSD-3 / Apache-2.0) and is fine-tuned on Berlin. The eyes are borrowed; the map still lives entirely in the weights.
The result is one file of 31.7 MB that answers in
48.8 ms . Nothing is looked up, nothing is matched
— the map is the weights. Each refinement on the way is its own
experiment in the research log , with the
figures on the model designs
page.
The $10 in the question is the module it is sized for:
an ESP32‑P4 with 16 MB of flash and 32 MB of memory, such as
the Waveshare ESP32-P4-Module ($9.99 retail; less in production
quantities), on a carrier board that brings out the camera connector. The
cheapest complete development board with the same memory and a camera
connector, the Olimex ESP32‑P4‑DevKit, costs €16. The file
fits that flash once its weights are stored as 8-bit integers instead of
32-bit floats, about a quarter of the size. That conversion, and running
the model on the chip itself, have not been tested yet.
6 research eras
109 experiments
$487.87 total cost
One model per area, by design. Asked about a block of Berlin held
out of training entirely, the same model returns usable fixes on 0.5% of frames and is confidently wrong on 21% — the map lives in
the weights, so ground it was never shown is ground it cannot place, and it
does not reliably know when it is there. The pipeline takes any bounding box; each trained model
knows only its own. Night flying is scoped out for now: everything here is
raw daytime imagery.
Brandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 280 of 2,970 52.52377 N · 13.32644 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 316 of 2,970 52.52471 N · 13.39433 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 1,059 of 2,970 52.50790 N · 13.34402 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 1,086 of 2,970 52.50861 N · 13.39492 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 1,792 of 2,970 52.49342 N · 13.37850 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 2,443 of 2,970 52.47938 N · 13.36205 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 2,562 of 2,970 52.47732 N · 13.37909 EBrandenburg DOP20 TrueDOP 2022 TrueDOP 2023 tile 2,571 of 2,970 52.47755 N · 13.39605 EEight of the 2,970 answers, each as it appears in the three training photographs, taken in different years and seasons; the model learns the place from all three and is tested on a fourth it never sees. Every tile is 128 m of real Berlin and owns one coordinate, so “which tile is this?” is the same question as “where am I?” — a river bend is easy; the empty field at the end is why the model must also know when not to answer.
4
How our research works The model was found, not designed — by a loop of coding agents.
INFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract the experiment goes here architecture · feature extraction · decode · confidence — the agent may redraw all of it each figure below is one proposal for the inside of this box whatever fills the box must export to one ONNX file ≤ 32 MiB (≈ 8 MiB int8) and answer in ≤ 250 ms frozen contract (lat, lon, confidence) position fix + confidence losses · supervision targets · samplers scaffolding that shapes the weights during training — torn down before flight, never in the exported model
The search space. The gray endpoints are frozen
— one camera crop in, one (lat, lon, confidence)
answer out — and the dashed box is everything an experiment may
change. Every design in the
model designs gallery is one way
of filling it.
No human designed the champion. A
Karpathy-style
autoresearch loop did: each round, a coding agent reads the full
experiment history, proposes one focused change, and pre-registers
its hypothesis and expected outcome before any code is written. The harness
trains it, scores it, and keeps it only if it beats the champion.
Two roles, split clean. The agents own the experiments. The
frozen harness owns the exam: the data pipeline, the held-out
viewpoints and the scorer are read-only, so the loop can rewrite the model
but never the ruler it is measured by. We supervise from outside
— set the levers, curate the inspiration, and decide when an era is
over.
THE LOOP
experiment after experiment
PROPOSE
one focused change
1
2
TRAIN
on the whole city
3
SCORE
the mission penalty
4
KEEP
only if it beats the champion
HUMAN SUPERVISOR
sets the levers,
curates the inspiration
FROZEN RULER
pipeline, eval set, scorer —
read-only to the agents
Each round keeps only what beats the best so far, and
the next experiment builds on it.
Pre-registered, one change at a time
Hypothesis, method and expected outcome are written down before
implementation, and each experiment makes exactly one focused change —
so every verdict in the record says something.
Pivoting on patience
A run has a patience : spend it on reverted experiments and the
next design must abandon the current backbone for a genuinely different one
— enforced by reading the resulting source, not by asking nicely.
A frozen ruler
The score is the product requirement — usable fixes minus
confidently-wrong ones — and the agent cannot touch it. When a score
and the product disagree, the ruler gets fixed and the lineage is re-measured,
never quietly erased.
Everything is recorded
Every experiment is a git commit, a database row and a gallery entry
— kept and reverted alike. The
lineage , evolution graph
and lab notebook are rendered from
that record, not written after the fact.
Our model designs
Every experiment's model design, drawn by the design agent itself
before it was allowed to train — camera frame entering on the
left, (lat, lon, confidence) leaving on the right, red marking what that
experiment changed. All 87 that carry a figure, kept and reverted
alike; the model designs page
has each at full size:
INFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 288²×3 resize to 288² bilinear, in-graph 0 params · was 224² 24×72² 40×36² 112×18² 320×9² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×18², 320×9² cosine tile match 112-d vs 2,970 protos, s=30 24×72² 40×36² 112×18² 320×9² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×18², 320×9² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains unchanged from exp 20 MixStyle · CosFace m=0.25 · trunk at 0.1× LR paired three-survey batches · re-photographing aug. 8 epochs · weight averaging from ep. 5 + BN recal. only the in-graph enlargement changed (224² → 288²) · parameters, size (31.7 MB), decode and gate identical to exp 20 6.28 · reverted · Bigger magnifying glassINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains view, survey a same ground, survey b composite view label unchanged NEW — patchwork surveys 60% of views: up to 3 rectangles, 12–48 px a side ¾ refilled from another year at the same ground, ¼ from unrelated ground (another crop in the batch) recognise the place from what survives local change unchanged from exp 20 MixStyle · CosFace m=0.25 · trunk at 0.1× LR weight averaging + BN recalibration BN recal. sees whole, unedited frames only the training views changed · graph, size (31.7 MB), decode and gate identical to exp 20 6.27 · reverted · Patchwork surveysINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weights averaged over epochs 5–8 · MixStyle off at export only the training schedule changed · graph, size (31.7 MB), decode and gate identical to exp 20 0 1 2 3 4 5 6 7 8 epochs fine-tune 1–8 as before 0 = head-only probe NEW — probe first, then fine-tune (LP-FT) extra epoch 0: trunks A, B held still — weights and BN stats stay ImageNet’s; only lifts, prototypes, offset head train the random head cannot distort borrowed eyes 6.26 · reverted · Probe first, then fine-tune, actually trainedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A · Noisy Student NEW init weights · TF-same pad, eps 1e-3 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B · Noisy Student NEW init weights · TF-same pad, eps 1e-3 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weight averaging epochs 5–8, BN stats re-measured 8 epochs · MixStyle is identity at export only the trunks' starting weights changed · graph, size (~31.7 MB), decode and gate identical to exp 20 300M noised photos NEW — Noisy Student trunk weights tf_efficientnet_b0.ns_jft_in1k, Apache-2.0 student trained under RandAugment, dropout, stoch. depth ported to torchvision · numerically checked vs timm init only — same graph, same size 6.25 · reverted · Same eyes, more worldly upbringing, actually trainedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A · Noisy Student NEW init weights · TF-same pad, eps 1e-3 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B · Noisy Student NEW init weights · TF-same pad, eps 1e-3 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weight averaging epochs 5–8, BN stats re-measured 8 epochs · MixStyle is identity at export only the trunks' starting weights changed · graph, size (~31.7 MB), decode and gate identical to exp 20 300M noised photos NEW — Noisy Student trunk weights tf_efficientnet_b0.ns_jft_in1k, Apache-2.0 student trained under RandAugment, dropout, stoch. depth ported to torchvision · numerically checked vs timm init only — same graph, same size 6.24 · reverted · Same eyes, more worldly upbringingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weight averaging epochs 5–8, BN stats re-measured 8 epochs · MixStyle is identity at export only the training loss and batching changed · graph, size (31.7 MB), decode and gate identical to exp 20 p₁ p₂ p₃ of one place → mixture p̄ NEW — same place, same belief (JSD, λ = 6) batches of 21 places × all 3 surveys (63 views) per member: tile-belief maps of one place in the three surveys pulled to their mean, ⅓ Σ KL(pk ‖ p̄) deployed margin-free softmax · no cross-member term 6.23 · reverted · Same place, same beliefINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weight averaging epochs 5–8, BN stats re-measured 8 epochs · MixStyle is identity at export only where training views stand changed · graph, size (31.7 MB), decode and gate identical to exp 20 ● lattice · × jittered vantages NEW — continuous vantages, ±12 px jitter fresh (dx, dy) ∈ [−12, 12]² px per use of a spot one shift shared by all three surveys, like heading tile and offset targets follow the shifted centre views tile the ground, not the fixed 24 m lattice 6.22 · reverted · Continuous vantagesINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss weights averaged over epochs 5–8 · MixStyle off at export only the training schedule changed · graph, size (31.7 MB), decode and gate identical to exp 20 0 1 2 3 4 5 6 7 8 epochs fine-tune 1–8 as before 0 = head-only probe NEW — probe first, then fine-tune (LP-FT) extra epoch 0: trunks A, B held still — weights and BN stats stay ImageNet’s; only lifts, prototypes, offset head train the random head cannot distort borrowed eyes 6.21 · reverted · Probe first, then fine-tuneINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle · CosFace m=0.25 · trunk at 0.1× LR — unchanged paired three-survey batches, re-photographing aug., offset loss 8 epochs · MixStyle is identity at export only which weights fly changed · graph, size (31.7 MB), decode and gate identical to exp 19 θ̄ = mean of visited θ NEW — weight averaging (SWA/SWAD) uniform mean, epochs 5–8, every 4 steps ~3.4k snapshots, both members A and B BN stats re-measured on 400 batches the average flies, not the last step 6.20 · champion · Fly the average, not the last stepINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 40×28² 112×14² 320×7² 112 belief field A NEW — EfficientNet-B0 trunk A 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 40×28² 112×14² 320×7² 112 belief field B NEW — EfficientNet-B0 trunk B 7 MBConv+SE stages · GAP of 112×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle after stages 2, 3, 4 (same strides) · CosFace m=0.25 · trunk at 0.1× LR — unchanged from exp 16 paired three-survey batches, re-photographing augmentation, offset loss; MixStyle is identity at export training views pass the same in-graph 128→224 resize · bf16 autocast on CUDA only (infrastructure) only the trunk changed: ImageNet EfficientNet-B0 (Apache-2.0) · decode, gate, losses unchanged 6.19 · kept · Sharper borrowed eyesINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 32×28² 96×14² 320×7² 336 belief field A MobileNetV2 trunk A 17 blocks · 96×14², 320×7² → 56 ch cosine tile match 336-d vs 2,970 protos, s=30 NEW — concentric-ring pooling centre · middle · edge, replaces the global average 24×56² 32×28² 96×14² 320×7² 336 belief field B MobileNetV2 trunk B 17 blocks · 96×14², 320×7² → 56 ch cosine tile match 336-d vs 2,970 protos, s=30 NEW — concentric-ring pooling centre · middle · edge, replaces the global average averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle after blocks 3, 6, 10 · CosFace m=0.25 · trunk at 0.1× LR — unchanged from exp 16 paired three-survey batches, re-photographing augmentation, offset loss; MixStyle is identity at export ring masks are fixed buffers, not trained · training views pass the same in-graph 128→224 resize only the pooling changed: global average → 3 concentric rings (heading-invariant, 0 params) · 112-d → 336-d 6.18 · reverted · Ring pooling, actually trainedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 32×28² 96×14² 320×7² 336 belief field A MobileNetV2 trunk A 17 blocks · 96×14², 320×7² → 56 ch cosine tile match 336-d vs 2,970 protos, s=30 NEW — concentric-ring pooling centre · middle · edge, replaces the global average 24×56² 32×28² 96×14² 320×7² 336 belief field B MobileNetV2 trunk B 17 blocks · 96×14², 320×7² → 56 ch cosine tile match 336-d vs 2,970 protos, s=30 NEW — concentric-ring pooling centre · middle · edge, replaces the global average averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle after blocks 3, 6, 10 · CosFace m=0.25 · trunk at 0.1× LR — unchanged from exp 16 paired three-survey batches, re-photographing augmentation, offset loss; MixStyle is identity at export ring masks are fixed buffers, not trained · training views pass the same in-graph 128→224 resize only the pooling changed: global average → 3 concentric rings (heading-invariant, 0 params) · 112-d → 336-d 6.17 · reverted · Ring poolingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 resize to 224² bilinear, in-graph 0 params · ImageNet's scale 24×56² 32×28² 96×14² 320×7² 112 belief field A NEW — MobileNetV2 trunk A 17 blocks · GAP of 96×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 24×56² 32×28² 96×14² 320×7² 112 belief field B NEW — MobileNetV2 trunk B 17 blocks · GAP of 96×14², 320×7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle after blocks 3, 6, 10 (same strides) · CosFace m=0.25 · trunk at 0.1× LR — unchanged from exp 15 paired three-survey batches, re-photographing augmentation, offset loss; MixStyle is identity at export training views pass the same in-graph 128→224 resize · bf16 autocast on CUDA only (infrastructure) only the trunk changed: ImageNet MobileNetV2 (BSD-3) · decode, gate, losses unchanged 6.16 · kept · Better borrowed eyesINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame ~1 m/px crop frozen contract 128² 224²×3 NEW — resize to 224² bilinear, in-graph 0 params · ImageNet's scale 16×112² 24×28² 48×14² 96×7² 112 belief field A trunk A (pretrained), read at 2 depths 11 MobileNetV3-S blocks · GAP of 14², 7² cosine tile match 112-d vs 2,970 protos, s=30 16×112² 24×28² 48×14² 96×7² 112 belief field B trunk B (pretrained), read at 2 depths 11 MobileNetV3-S blocks · GAP of 14², 7² cosine tile match 112-d vs 2,970 protos, s=30 averaged belief p̄ mean of both softmaxes pooled 3×3 centroid + mean offset nudge frozen contract (lat, lon, confidence) position fix + confidence confidence = min(mA, mB) both members must back the 3×3 block else the UAV abstains MixStyle after blocks 1, 3, 6 · CosFace margin m=0.25 · trunk at 0.1× LR — unchanged from exp 14 paired three-survey batches, re-photographing augmentation, offset loss; MixStyle is identity at export NEW (by construction) — training views pass the same in-graph 128→224 resize, so train and flight see one scale trunk weights, decode, gate, losses unchanged · ONNX I/O still 1×3×128² 6.15 · kept · Look at Berlin at the borrowed eyes' native scaleINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY
128²×3 camera frame ~1 m/px crop frozen contract 32²×16 16²×24 8²×40 8²×48 4²×96 member 1 112 ~3,000 tiles 32²×16 16²×24 8²×40 8²×48 4²×96 member 2 112 ~3,000 tiles pretrained trunk · 11 blocks MobileNetV3-Small, ImageNet init, 0.1× LR place description two depths lifted to 56, averaged tile belief cosine × 30 vs tile directions 3×3 mass within-cell offset red ◇: style mixing, training only place description same design, own weights tile belief 3×3 mass confidence = min of both averaged belief 3×3 balance point local belief pooling frozen contract (lat, lon, confidence) position fix + confidence NEW · cross-survey style mixing (MixStyle) after blocks 1, 3, 6, p = 0.5: per-channel μ, σ mixed with another survey's view, λ ~ Beta(.1, .1) layout untouched · both members · off in flight angular-margin tile loss on each member's beliefs paired three-survey batches, re-photographing augmentation offset loss within-cell target 6.14 · kept · Cross-survey style mixing inside the borrowed eyesINFERENCE PATH — WHAT FLIES
Camera frame 128²×3 crop, ~1 m/px frozen contract 64²×16 32²×16 16²×24 8²×48 4²×96 2×56 112 ~3,000 tiles 8² tap 128²×3 Feature extractor — ImageNet-pretrained MobileNetV3-Small ×2 stem + 10 inverted-residual blocks (BSD-3), read at two depths: 8²×48 (texture) and new 4²×96 (layout), each 1×1-lifted to 56, pooled, concatenated Map-cell classifier cosine vs one unit vector per tile, ×30 member 2 — independent copy 112 descriptor Local belief pooling mean of the 2 maps, 3×3 balance point min Within-cell offset 2-d nudge per member, averaged Confidence gate smaller 3×3 belief mass of the two frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY all 11 pretrained blocks fine-tuned at 0.1× the head LR, same cosine decay new lifts train at full LR with the heads angular-margin tile loss + offset loss, paired three-survey batches 6.13 · kept · Deeper borrowed eyesINFERENCE PATH — WHAT FLIES
Camera frame 128²×3 crop, ~1 m/px frozen contract 64²×16 32²×16 16²×24 8²×48 8²×112 112 ~3,000 tiles 128²×3 Feature extractor — ImageNet-pretrained MobileNetV3-Small ×2 stem + 8 inverted-residual blocks (BSD-3), then 1×1 lift 48→112 Map-cell classifier cosine vs one unit vector per tile, ×30 member 2 — independent copy 112 descriptor Local belief pooling mean of the 2 maps, 3×3 balance point min Within-cell offset 2-d nudge per member, averaged Confidence gate smaller 3×3 belief mass of the two frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY trunk fine-tuned at 0.1× the head LR (1e-4 → 1e-5), same cosine decay so Berlin is added to ImageNet invariances, not written over them angular-margin tile loss + offset loss, paired three-survey batches 6.12 · kept · Borrowed eyesINFERENCE PATH — WHAT FLIES
128²×3 camera frame ~1 m/px crop frozen contract member A · conv trunk 3×3 s2 ×4 · BN+ReLU · own weights 64²×24 8²×112 112 GAP 2970 30·cos θ tile belief p₁ softmax · 2970 tiles cosine head: one direction per tile BN → ‖e‖ = 1 · cos θ to unit prototypes (no bias) member B · conv trunk 3×3 s2 ×4 · BN+ReLU · own weights 64²×24 8²×112 112 GAP 2970 30·cos θ tile belief p₂ softmax · 2970 tiles mean argmax tile + its 8 neighbours averaged belief p̄ → block decode block centroid + mean of the two offsets m₁ = p₁ mass on the consensus block m₂ = p₂ mass on that block confidence = min(m₁, m₂) both members must back the block; if either looks elsewhere → abstain frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY paired three-survey batches nuisance augmentation, unchanged additive margin (CosFace), training only logits = 30·(cos θ − 0.25·t̂) on true tile + tent → every survey's view must share one direction Σ members: soft-tent tile CE + smooth-L1 offset loss, unchanged 6.11 · kept · One direction per tileINFERENCE PATH — WHAT FLIES 128²×3 camera frame ~1 m/px crop frozen contract member A trunk 4 strided convs 8²×112 112-d GAP FC → 2,970 logits belief field A softmax, 2,970 map cells member B trunk 4 strided convs 8²×112 112-d GAP FC → 2,970 logits belief field B softmax, 2,970 map cells two members, no shared weights averaged belief mean of both softmaxes local belief pooling centroid of winning tile + 8 neighbours + mean of the two members' offset nudges confidence gate min of the two members' 3×3-block mass both must back the same place, or the UAV abstains frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY 8²×112 32²×3 decoder B — training-only 3×3 conv · 2× up, 112→64→32→3 8²×112 32²×3 decoder A — training-only 3×3 conv · 2× up, 112→64→32→3 MSE × 3, each member target crop 32²×3 · 4 m/px standardised per crop cut at view pose, pool 4× median composite per-pixel median, built once 3 training surveys bb_dop20 · TrueDOP 2022 · 2023 colour-normalised; cars, shadows, one-off canopy vanish in the median 2024 eval photograph never opened plus, unchanged: each member's own tile tent-CE + offset loss · decoders dropped before export 6.10 · reverted · Canonical-composite reconstructionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame Berlin crop, ~1 m/px frozen contract two members, no shared weights member A trunk strided conv stack 128²×3 8²×112 feature f 112-d GAP FC → 2970 logits + offset belief field A 128 m map cells member B trunk strided conv stack 128²×3 8²×112 feature f 112-d GAP FC → 2970 logits + offset belief field B 128 m map cells mean averaged belief p̄ local 3×3 pooling pooled centroid + mean within-cell offset confidence gate min(mA, mB) on the 3×3 block may abstain frozen contract (lat, lon, confidence) position fix + confidence DOP20 2022 2023 same place, same heading f in 3 surveys NEW · within-place variance penalty, each member, weight 10 standardise every one of the 112 dims over the batch, then penalise the spread of one place's 3 survey features: 0 = identical, 1 = as far apart as two places added to each member's own cell + offset loss; batches = 21 places × 3 surveys deployed graph byte-identical · 2024 survey never opened in training 6.9 · reverted · Within-place variance penaltyINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY
128²×3 camera frame one daytime crop frozen contract member 1 · own weights member 2 · own weights 64²×24 8²×112 112 offset 112→2 conv encoder, ×2 4× conv 3×3, s2 · not shared GAP → FC heads 112-d → tile logits + offset tile belief ~3,000 × 128 m mean belief of the two members local belief pooling balance point of the 3×3 within-cell offset + mean of the 2 nudges confidence gate min over the members of their own 3×3 mass frozen contract (lat, lon, confidence) position fix + confidence adapter, 1×1 conv 112→512→384 · discarded student map 8²×384, projected 1 − cos ×5, every cell teacher tokens 8²×384, batch-centred DINOv2 ViT-S/14 frozen, 22M · tutor only 112²×3 same spot & heading OTHER survey, clean NEW · cross-survey distillation each member's 8×8 map, seen on its own augmented crop, must say what DINOv2 says about a DIFFERENT survey's photo of it; tutor + adapter never exported tile CE on a tent target + offset loss per member, unchanged 6.8 · reverted · Cross-survey DINOv2 distillation
INFERENCE PATH — WHAT FLIES
128²×3
camera frame
~1 m/px crop
frozen contract
member A · conv trunk
3×3 s2 ×4 · BN+ReLU · own weights
64²×24
8²×112
112
GAP
FC → 2970 logits
softmax · + offset FC → 2
tile belief p₁
2970 tiles
member B · conv trunk
3×3 s2 ×4 · BN+ReLU · own weights
64²×24
8²×112
112
GAP
FC → 2970 logits
softmax · + offset FC → 2
tile belief p₂
2970 tiles
mean
argmax tile + its 8 neighbours
averaged belief p̄ → block decode
block centroid + mean of the two offsets
m₁ = p₁'s mass on the consensus block
m₂ = p₂'s mass on that block
confidence = min(m₁, m₂)
both members must back the block;
if either looks elsewhere → abstain
frozen contract
(lat, lon, confidence)
position fix + confidence
TRAINING SIGNALS — NEVER FLY
each member: its OWN augmentation draw
of the same crops · brightness, contrast, blur, grain
+ its own random init
loss = Σ over members (soft-tent tile CE + offset loss)
no agreement term — diversity is the mechanism
6.7 · kept · Two-member committeeINFERENCE PATH — WHAT FLIES 128²×3 camera frame ~1 m/px daytime crop frozen contract feature extractor strided conv stack · BN + ReLU …×160 160-d pooled ~3,000 map-cell classifier FC → ~3,000 · softmax map-cell belief 128 m tiles of Berlin local belief pooling balance point of winner + 8 neighbours within-cell offset small head → (Δu, Δv) nudge confidence gate mass on winner + 8 neighbours low → abstain frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY bb_dop20 2022 2023 three surveys · same place, same heading × weight field 4² logits ↑128², softmax = cross-survey blend per-pixel convex mix · p = 0.75 then photometric · blur · grain re-photographing, unchanged cross-entropy on bilinear tent cell target + within-cell offset loss · labels unchanged 6.6 · reverted · Cross-survey blendingINFERENCE PATH — WHAT FLIES
128²×3 camera frame ~1 m/px daytime crop frozen contract 64²×32 8²×160 2²×160 convolutional encoder Conv 3×3 s2 ×4 · BN+ReLU · RF 31 px +2 depthwise-sep. s2 dw 3×3 + pw 1×1 · +55k RF 127 px = whole frame 160 160-d GAP 2×2 2,970 FC → cell logits + offset head FC → 2 cell belief field softmax over 2,970 tiles 3×3 belief pooling + offset balance point of winner + 8 neighbours frozen contract (lat, lon, confidence) position fix + confidence confidence = 3×3 mass may abstain TRAINING SIGNALS — NEVER FLY every view re-photographed at random (unchanged) brightness · contrast · gamma · colour · blur · grain; each batch = same places in 3 surveys bilinear tile CE + residual offset (unchanged) cosine schedule · views/epoch as exp. 4 6.5 · reverted · Whole-frame receptive fieldINFERENCE PATH — WHAT FLIES
Camera frame 128²×3 crop, ~1 m/px frozen contract 128²×3 8²×160 Feature extractor strided conv stack, last layer 160 channels 160 ~3,000 tiles of 128 m Map-cell classifier one score per tile Local belief pooling balance point of tile + 8 neighbours Within-cell offset small 2-d nudge head Confidence gate belief mass on 3×3 frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY brighter darker warmer softer grainier NEW — every training view re-photographed at random, independently per view: exposure 0.75–1.30 × colour gain 0.85–1.15 · contrast 0.6–1.4 · brightness ±0.12 · gamma 0.7–1.4 · saturation 0.5–1.5 · blur σ ≤ 1.5 px (p 0.7) · sensor noise σ ≤ 0.03 so one survey's pixel fingerprint no longer names the tile — only shared street layout does tile loss + offset loss, paired 3-survey batches (unchanged) 6.4 · kept · Exposure-and-sharpness nuisance randomisationINFERENCE PATH — WHAT FLIES 128²×3 camera frame ~1 m/px crop frozen contract ×160 ch feature extractor strided conv 3×3 stack · BN + ReLU 160-d feature f global avg pool ~2,970 map-cell classifier FC 160 → ~2,970 tiles · softmax tile belief + 3×3 window winning 128 m tile and its 8 neighbours local belief pooling balance point of the 3×3 mass within-cell offset FC 160 → 2 · tanh ×½ nudge confidence gate mass on the 3×3 window scattered belief → abstain frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY unit sphere DOP20 TrueDOP 2022 TrueDOP 2023 same place, same heading NEW · cross-survey feature consistency each survey's normalised f is pulled toward the stop-gradient mean of the other two at the same place: ×2.0 · (1 − cos) batches = 21 places × 3 surveys · eval photo 2024 never seen soft cross-entropy on tile labels + offset regression (unchanged) 6.3 · reverted · Cross-survey latent consistencyINFERENCE PATH — WHAT FLIES 128²×3 camera frame one daytime view frozen contract 64²×24 8²×160 feature extractor strided conv 3×3 stack · BN + ReLU 160 ~3,000 map-cell classifier FC → one logit per 128 m tile belief over tiles local belief pooling winning tile + its 8 neighbours → balance point within-cell offset Δ(u, v) nudge confidence gate 3×3 mass — low ⇒ abstain frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY bb_dop20 TrueDOP 2022 TrueDOP 2023 mixed batch NEW · paired-survey batches: each position is cut from ALL THREE photographs one shared heading, one tile label, shuffled together — no batch is single-survey, so an epoch no longer overwrites two surveys with the third (views/epoch unchanged: 218,136) loss unchanged: soft-target CE over tiles + offset loss 6.2 · kept · Paired-survey batchesINFERENCE PATH — WHAT FLIES 128²×3 camera frame ~1 m/px crop frozen contract 64²×32 8²×160 convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 160-d global avg pool cell classifier Linear → 2,970 within-cell offset Linear 160 → 2 · tanh·½ softmax map-cell belief 55×54 · 128 m tiles 3×3 belief pooling mass-weighted centroid nudge (Δu, Δv) inside the cell confidence gate Σ mass on the 3×3 block or abstain frozen contract (lat, lon, confidence) position fix + confidence TRAINING SIGNALS — NEVER FLY 3 training photographs DOP20 · TrueDOP 2022 · 2023, full lattice, rotation only TrueDOP 2024 held out — never read in training soft-label cross-entropy bilinear tent over the nearest cells smooth-L1 on residual offset the error the pooled centroid leaves 6.1 · kept · Baseline seedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×32 8²×160 160-d GAP FC → 2,970 cell logits softmax probability field 55×54 map cells · 128 m each 3×3 pooled centroid mass-weighted balance point of the winning cell + 8 neighbours within-cell offset tanh · ½ cell frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = pooled mass over the whole 3×3 block may abstain instead of guessing soft cross-entropy vs a bilinear tent over the 4 nearest cell centres — border views taught the split their decode produces offset target = the residual the pooled centroid leaves (±½ cell), not the position that centroid already encodes deployed graph byte-identical to Fig. 4 — only the training targets changed 5.5 · kept · Decode-consistent training targetsINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×32 8²×160 160-d GAP FC → 2,970 cell logits softmax probability field 55×54 map cells · 128 m each 3×3 pooled centroid mass-weighted balance point of the winning cell + 8 neighbours within-cell offset tanh · ½ cell frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = pooled mass over the whole 3×3 block may abstain instead of guessing cross-entropy on the one true cell + smooth-L1 offset — unchanged from Fig. 3 full-lattice coverage, fresh rotations, cosine LR — byte-identical to Fig. 3; only the deployed decode moved 5.4 · kept · Local belief pooling decodeINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×32 8²×160 160-d GAP FC → 2,970 cell logits softmax probability field 55×54 map cells · 128 m each argmax over 2,970 cells one winning cell → its centre, nudged by the offset within-cell offset tanh · ½ cell frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = softmax mass on the winning cell may abstain instead of guessing cross-entropy + offset loss unchanged — but every epoch is now one FULL pass over all 72,712 lattice positions 581,696 views, ~196 per cell (12× Fig. 2) · fresh rotation per position per epoch · cosine LR 1e-3 → 1e-4 architecture restored verbatim from Fig. 2 — the training schedule is the change 5.3 · kept · Full-lattice coverage training for the map-cell classifierINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×32 8²×160 160-d GAP FC → 2,970 cell logits softmax probability field 55×54 map cells · 128 m each argmax over 2,970 cells one winning cell → its centre, nudged by the offset within-cell offset tanh · ½ cell frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = softmax mass on the winning cell may abstain instead of guessing cross-entropy on the true cell (label smoothing 0.05) + smooth-L1 on the within-cell offset 6,000 of the 72,712 lattice positions redrawn each epoch with fresh headings — 48,000 views for 2,970 cells (~16 each) 5.2 · reverted · Map-cell classification decodeINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 3 sigmoid one shot — no map structure frozen contract (lat, lon, confidence) position fix + confidence mean-squared error on (u, v) the whole map supervises one number pair BCE ×0.1 on confidence target: error < half the map extent starting line — the harness's own code, no design agent involved 5.1 · kept · Baseline seedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 1,024 cell logits + 1 confidence softmax probability field 1,024 map cells · 32×32 · 217 m each argmax over 1,024 cells one winning cell → its centre, nudged by its own offset within-cell offset FC 128 → 1,024×(du,dv) · tanh ±½ cell frozen contract (lat, lon, confidence) position fix + confidence BCE ×0.1 on confidence the baseline's free sigmoid, untouched cross-entropy on the true cell + MSE on its own offset teacher-forced; trunk, data and 8-epoch budget unchanged drawn identically on purpose — a faithful rebuild of the experiment whose source a bad metric deleted 4.4 · kept · Re-implementation of lost experiment 2 — classify-then-refineINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 3 sigmoid one shot — no map structure frozen contract (lat, lon, confidence) position fix + confidence every presentation is now a different vantage at a fresh heading ~48,000 distinct lattice positions streamed at the same 48,000 gradient steps loss unchanged MSE on (u, v) + BCE ×0.1 on confidence the graph is the baseline's, untouched — the data diet is the whole experiment 4.3 · reverted · Coverage over repetitionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 1,024 cell logits + 1 confidence softmax probability field 1,024 map cells · 32×32 · 217 m each argmax over 1,024 cells one winning cell → its centre, nudged by its own offset within-cell offset FC 128 → 1,024×(du,dv) · tanh ±½ cell frozen contract (lat, lon, confidence) position fix + confidence BCE ×0.1 on confidence the baseline's free sigmoid, untouched cross-entropy on the true cell + MSE on its own offset teacher-forced; trunk, data and 8-epoch budget unchanged 4.2 · kept · Classify-then-refineINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 3 sigmoid one shot — no map structure frozen contract (lat, lon, confidence) position fix + confidence mean-squared error on (u, v) the whole map supervises one number pair BCE ×0.1 on confidence target: error < half the map extent starting line - the same naive baseline, re-measured on the corrected viewpoint holdout 4.1 · kept · Baseline seed on the corrected viewpoint-holdout splitINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 4,096 cell logits softmax probability field 4,096 map cells · 64×64 · 108 m each soft-argmax over 4,096 cells every cell centre, weighted by its probability frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = heaviest 5×5-cell block of belief may abstain instead of guessing loss, head and schedule all unchanged — what moves is which 6,000 positions each pass is built from re-drawn at random from all ~45,000 mapped vantages before every pass, instead of one frozen 6,000 draw reused 96 times deployed graph byte-identical to the experiment above it — only the training diet changed 3.5 · kept · Per-epoch position resamplingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 4×4×32 descriptor 1×1 conv 128→32 · 2×2 pool FC 512 → 256 cell logits softmax probability field 256 map cells · 16×16 · 434 m each posterior-weighted blend every cell's centre plus its own pointer, averaged by belief per-cell pointer FC 512 → 256×(du,dv) · tanh ±¾ cell frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = heaviest 2×2-cell block of belief may abstain instead of guessing soft-label cross-entropy over the 256 coarse cells (σ = ½ cell) + MSE on the true cell's pointer + MSE on the blended (u, v) same 96-pass, fresh-heading schedule as the champion — the loss now grades 256 big squares and teaches a pointer inside each the descriptor moves with the decode: a pooled texture bag cannot say WHERE in a cell the frame sits 3.4 · reverted · Hierarchical decodeINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 4,096 cell logits softmax probability field 4,096 map cells · 64×64 · 108 m each soft-argmax over 4,096 cells every cell centre, weighted by its probability frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = heaviest 5×5-cell block of belief may abstain instead of guessing same cross-entropy and same soft labels — the study budget is the change: 96 passes, not 8 cosine LR 1e-3 → 1e-5 · every crop re-drawn at a fresh random heading before every pass the head is restored verbatim from the gated-out experiment above it — only the training schedule is new 3.3 · kept · Re-land the validated 64x64 grid-classification head with a convergence-scale training budget (x12 epochs, cosine LR, fresh per-epoch rotations)INFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 4,096 cell logits softmax probability field 4,096 map cells · 64×64 · 108 m each soft-argmax over 4,096 cells every cell centre, weighted by its probability frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 = heaviest 5×5-cell block of belief may abstain instead of guessing cross-entropy against Gaussian-smoothed cell labels (σ = 1 cell) + a small L2 on the soft-argmax coordinate 6,000 crops, one frozen rotation each, 8 epochs — the same budget the naive baseline was given gated FAIL — confident on only 10.8% of frames, under the harness's 20% coverage floor 3.2 · reverted · Reparameterize localization as 64x64 map-grid classification with in-graph soft-argmaxINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one daytime frame frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 3 sigmoid one shot — no map structure frozen contract (lat, lon, confidence) position fix + confidence mean-squared error on (u, v) the whole map supervises one number pair BCE ×0.1 on confidence target: error < half the map extent starting line — the harness's own code re-run on Berlin alone, no design agent involved 3.1 · kept · Baseline seedINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract ×8 exact-pixel views (D4) rotations + mirror, shared weights SqueezeNet1.1 Fire trunk stem + 4 fire blocks (BSD-3) mobilenet_v3_small banned this round 63²×64 7²×384 lum FiLM (γ, β) MLP modulates feature map 384-d descriptor GAP, D4-pooled avg 1-ch layout 1×1 squeeze, 49-d flat single Linear → 1,024 gate + dual head deleted probability field 32×32 cells over the map adaptive-β commit frozen contract (lat, lon, confidence) position fix + confidence β = f(entropy, peak) learned per-example sharpening confidence 0–1 384-d GAP + adaptive β + peak/entropy/gap Gaussian-CE vs cell target (unchanged) σ = 1.5 cells over the 32×32 field L2 on decoded (u, v) expected-coordinate loss — now also trains the β gauge confidence BCE ×0.3 (unchanged loss) same hit-radius target, new head inputs 2.62 · reverted · Fire-module SqueezeNet1.1 trunk + FiLM-conditioned single field + D4 symmetry + adaptive-sharpening decode (forced pivot, mobilenet_v3_small banned)INFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract ×4 crop rotations (C4) 0/90/180/270°, shared weights depthwise-separable trunk from-scratch, 4× wider 64²×64 8²×320 320-d descriptor GAP, C4-pooled avg unified FC → 1600 logits QATLinear, weight fake-quant probability field 40×40 cells, finer than prior 32×32 sharpen β=3 → soft-argmax over the finer 1,600-cell grid frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 peak/entropy/gap of new field weight fake-quant every step round(w/scale)·scale — straight-through grad int8 export, post-training dynamic quantization — 1 B/weight, real rounding Gaussian-CE vs cell target (unchanged) σ = 1.5 cells, now over the 40×40 field L2 on decoded (u, v) expected-coordinate loss, unchanged confidence BCE ×0.3 unchanged hit-radius target 2.61 · reverted · Quantization-freed capacityINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract illumination-invariant channel log-luminance − own Gaussian blur (fixed math, no gradient) concat 128²×4 input ×4 crop rotations (C4) 0°/90°/180°/270°, shared weights from-scratch conv encoder Conv 3×3 s2 ×4 · BN+ReLU — pretrained trunk banned this round 128²×4 8²×128 128-d descriptor GAP, C4-pooled avg unified FC → 1024 logits gate + dark-expert head deleted probability field 32×32 cells, same layout-code concat sharpen β=3 → soft-argmax decode mechanism unchanged frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 peak/entropy/gap of new field same recipe, no gate-derived inputs left lighting-partner crop same spot & heading, other bucket train-only shared trunk f partner 2nd forward pass L2 consistency loss ×0.1 (new) same spot, 2 lighting buckets → match descriptors watch: across-location variance must stay healthy Gaussian-CE vs cell target (unchanged) σ = 1.5 cells, same GRID_K=32 field L2 on decoded (u, v) (unchanged) confidence BCE ×0.3 (unchanged loss) same hit-radius target, new field inputs 2.60 · reverted · Corrected exp-38 retryINFERENCE PATH — WHAT FLIES (both specialists always run; the gate blends, never branches) TRAINING — THREE PHASES, NEVER JOINT 128²×3 camera frame one of 6 lighting buckets frozen contract day-specialist trunk from-scratch · day buckets only day field + decode 16×16 grid, soft-argmax day (u,v,conf) 3 pixel stats mean · spread · dark% MLP(8) no trunk, no features gate g day-like ↔ night-like dispatcher night-specialist trunk independent copy, own weights · night buckets only night field + decode 16×16 grid, soft-argmax night (u,v,conf) blend g·day + (1−g)·night conf nudged down when g is unsure frozen contract (lat, lon, confidence) position fix + confidence phase 1 — day trunk+field+decode+conf fresh optimizer · Gaussian-CE + L2 + 0.3× conf BCE, day buckets only phase 2 — night trunk+field+decode+conf separate fresh optimizer · same loss · zero gradient crosses into phase 1 phase 3 — freeze both specialists train only the gate (~40 params) · Huber/L2 + 0.3× conf BCE, all 6 buckets confusability sampler runs twice, independently day draw and night draw never mix 2.59 · reverted · Independently-trained day/night specialist twins with a trunk-free brightness dispatcherINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — JOINT EVERY STEP, NEVER FLY 128²×3 camera frame one of 6 lighting buckets frozen contract rotation fan 4× C4 views, shared net convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×64 layout summary spatial code + texture avg FC → 1024 logits single unified scorer probability field 32×32 field, gate removed soft-argmax decode reads one unified field, sharpening unchanged frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 peak / entropy / gap + raw crop brightness (new) may abstain instead of guessing reconstruction decoder ConvTranspose 4×4 ×4 — reads row 0, un-rotated 8²×64 64²×16 repainted crop 128²×3, sigmoid discarded before export reference.tif twin same cx, cy, angle clean daytime, any bucket L1 pixel-reconstruction loss, weight 1.0 every crop, every bucket, every step — not a pretrain phase Gaussian-CE on field + L2 on (u,v) + BCE on confidence loss formula itself unchanged — only the field it grades is unified now one shared LR (1e-3) for all params from-scratch trunk — no pretrained-feature 10× discount to protect 2.43 · reverted · Joint reconstruction-canonicalization trunkINFERENCE PATH — WHAT FLIES PHASE 0 — SELF-SUPERVISED PRETRAINING (TRAIN SPLIT ONLY, RUNS FIRST) PHASE 2 — SUPERVISED TRAINING (UNCHANGED LOSS FORMULA, NEW SCHEDULE) 128²×3 camera frame one night exposure frozen contract C4 rotation fan shared net, feeds SSL views from-scratch conv trunk 4× conv3×3 s2 + BN + ReLU, no ImageNet weights 128²×3 8²×64 layout summary spatial code + texture avg FC → 1024 logits single unified scorer probability field 32×32 field, gate removed soft-argmax decode answer = Σ probability · cell-center confidence 0–1 peak/entropy/gap + brightness (new) may abstain instead of guessing frozen contract (lat, lon, confidence) position fix + confidence view A lighting/heading draw 1 view B lighting/heading draw 2 same physical location — uniform random draw, 128 locs/step shared trunk same weights as above proj head 64→128→64, L2-norm in loss same place → pull together other batch places → push apart NT-Xent (InfoNCE) temperature 0.1, batch 128 schedule: 150 contrastive steps (3×50), 128 locs/step, then usual supervised epochs (count unchanged) trunk LR gap to head relaxed 10× → 2× (5e-4 vs 1e-3) — pretrained trunk needs less protection Gaussian-CE on field + L2 on decoded (u,v) — loss formula itself unchanged confusability-weighted sampler (exp 35) still feeds THIS stage only, not pretraining 2.42 · reverted · Domain-native contrastive pretraining replaces the ImageNet trunk (corrected re-attempt)INFERENCE PATH — WHAT FLIES PHASE 0 — SELF-SUPERVISED PRETRAINING (TRAIN SPLIT ONLY, RUNS FIRST) PHASE 2 — SUPERVISED TRAINING (UNCHANGED LOSS FORMULA, NEW SCHEDULE) 128²×3 camera frame one night exposure frozen contract C4 rotation fan shared net, feeds SSL views from-scratch conv trunk 4× conv3×3 s2 + BN + GELU, no ImageNet weights 128²×3 8²×64 layout summary spatial code + texture avg FC → 1024 logits single unified scorer probability field 32×32 field, gate removed learned β softplus, init 3.0 soft-argmax decode answer = Σ probability · cell-center confidence 0–1 peak/entropy/gap + brightness (new) may abstain instead of guessing frozen contract (lat, lon, confidence) position fix + confidence view A lighting/heading draw 1 view B lighting/heading draw 2 same physical location — confusability-weighted sampler (exp 35) shared trunk same weights as above proj head 64→64→32, L2-norm same place → pull together other batch places → push apart NT-Xent (InfoNCE) temperature 0.15, batch 128 schedule: 3 SSL epochs (cosine→0), then 24 supervised epochs (count unchanged) trunk/head LR ratio relaxed 10× → 3.3× (3e-4 vs 1e-3) — domain prior needs less protection Gaussian-CE on field + L2 on decoded (u,v) — loss formula itself unchanged same confusability-weighted sampler (exp 35) feeds this stage too 2.41 · reverted · Domain-native contrastive pretraining replaces the ImageNet trunkINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract rotation fan ×4 turns, C4 vote feature extractor MobileNetV3-S, 2 taps ↓ stride-8 tap feeds the fine field below coarse localization field 8×8 cells ≈875 m, shared 1×1 conv fine offset field per-cell (Δu,Δv,logσ), ONE shared head decode β-sharpened mean of (cell center + own offset) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 peak/entropy shape + fine field's own logσ — may abstain coarse Gaussian-CE over 8×8 cells (was 32×32) gradient now shared through ~100 conv weights, not 1,024 free rows + fine offset Huber loss, gathered at the TRUE cell only unimodal regression — coarse stage already picked the place + L2 on final decode + conf BCE training data: confusability sampler (exp 35) recomputed at the new 8×8 granularity schedule: 2-epoch warmup, fresh offset head 2.40 · reverted · Coarse-to-fine geolocalizationINFERENCE PATH — WHAT FLIES 128²×3 Camera frame one of 6 lighting renders frozen contract 4×128²×3 Rotation fan votes the new similarity map 64²×16 8²×48 Feature extractor MobileNetV3-Small, +NEW stride-8 tap 16²×24 skip GAP · 24-d NEW tap GAP 48 + layout 512 kept descriptor pair Crop embedding 64-d, L2-normalized — NEW lighting FiLM γ,β from brightness+GAP Probability map cosine sim. × cells, C4-avg Fourier(u, v) shared 2-layer MLP generates all 1,024 cell embeddings — not 1,024 free rows Decode β-sharpened soft-argmax (kept form) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 peak-vs-2nd-peak margin — NEW training-only — never flies Gaussian-bump CE on the temperature-scaled cosine-sim field (kept CE form; logits are bounded sim × learned scale, not raw FC output) + coord L2 + conf BCE (kept, unchanged math) — training data/schedule below kept, unchanged: confusability-weighted location sampler (rd 35) kept, unchanged: 3 seeded relight realizations per bucket (rd 17) kept, unchanged: 24-pass fresh-draw epochs, cosine LR glide to zero (rd 25) removed: dark-expert head + brightness gate (rd 12) — folded into FiLM above 2.39 · reverted · Coordinate-generated map fieldINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract 6-ch input RGB ⊕ retinex illumination-invariant channel log(I) − large-σ Gaussian blur, fixed Rotation fan 4×90° turns, 6ch MobileNetV3-Small trunk (9 blocks) pretrained · stem widened, rest frozen 64²×6 8²×48 stem re-init 3→6ch RGB weights kept + 0.1× copy 48-d GAP texture avg, unchanged layout squeeze 1×1 conv, 8ch 512-d layout code NEW: L2-norm per cell probability map gated 2-expert blend, unchanged β-sharpened decode C4-voted, unchanged frozen contract (lat, lon, confidence) position fix + confidence confidence: field-shape gauge, unchanged training data: confusability draw (kept, exp 35) + NEW per-crop sharpness jitter 50% chance Gaussian blur (σ 0.3–1.5px) else unsharp-mask — the blur-riding shortcut can no longer carry location signal training schedule: NEW 5%-of-steps linear warm-up before the exp-25 cosine decay protects the freshly re-initialized stem weights from early large gradients training signal: Gaussian-CE + coord L2 + conf BCE unchanged 2.38 · reverted · Illumination-invariant retinex channel joins raw RGB from the pixel level through the trunkCamera frame 128²×3 · ~1 m/px frozen contract C4 rotation fan 4 exact 90° turns 4×128²×3 64²×16 16²×24 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned GAP 48 layout 512 Multi-hypothesis head 8 learned points, softmax weight no separate expert 4-view weighted vote weight-argmax point per heading, softmax-blended conf: 4-view agreement × winning weight (lat, lon, confidence) frozen contract training-only — never flies cell-descriptor map from reference.tif, offline, once per area confusability-weighted sampling far, look-alike cells drawn ~2x more often per epoch half the draw stays uniform — full coverage kept (kept · rd 35) still 6,000 places/bucket/epoch, fresh-drawn (rd 20 · reweighted only) 3 seeded relight realizations per bucket (kept · rd 17) 24 passes · cosine LR glide to zero (kept · rd 25) + winner-take-all: nearest of 8 hyps gets the L2 grad, others ε=0.05 (NEW) weight head trained by cross-entropy vs. the nearest-hypothesis index (8-way, not 1024-way) winner-take-all regression nearest hyp wins the gradient; rest share ε=0.05 2.37 · reverted · Multi-hypothesis coordinate regression replaces the 1024-cell fieldCamera frame 128²×3 · ~1 m/px frozen contract C4 rotation fan 4 exact 90° turns 4×128²×3 64²×16 16²×24 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned GAP 48 layout 512 lighting gate brightness-blended pair of scorers σ probability field 32×32 cells, C4-vote averaged committed decode β-sharpened soft-argmax (unchanged) conf: field-shape hit predictor (lat, lon, confidence) frozen contract training-only — never flies cell-descriptor map from reference.tif, offline, once per area confusability-weighted sampling far, look-alike cells drawn ~2x more often per epoch half the draw stays uniform — full coverage kept (kept · rd 35) still 6,000 places/bucket/epoch, fresh-drawn (rd 20 · reweighted only) 3 seeded relight realizations per bucket (kept · rd 17) 24 passes · cosine LR glide to zero (kept · rd 25) + hardest-impostor margin hinge on C4-voted logits (NEW) hardest-impostor margin hinge s_neg (Cheb>3) pushed below s_pos (Cheb≤1) 2.36 · reverted · Hardest-impostor margin hinge, retested on the confusability-weighted championCamera frame 128²×3 · ~1 m/px frozen contract C4 rotation fan 4 exact 90° turns 4×128²×3 64²×16 16²×24 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned GAP 48 layout 512 lighting gate brightness-blended pair of scorers σ probability field 32×32 cells, C4-vote averaged committed decode β-sharpened soft-argmax (unchanged) conf: field-shape hit predictor (lat, lon, confidence) frozen contract training-only — never flies cell-descriptor map from reference.tif, offline, once per area NEW — confusability-weighted sampling far, look-alike cells drawn ~2x more often per epoch half the draw stays uniform — full-map coverage kept still 6,000 places/bucket/epoch, fresh-drawn (rd 20 · reweighted only) 3 seeded relight realizations per bucket (kept · rd 17) 24 passes · cosine LR glide to zero (kept · rd 25) Gaussian-bump CE + coord L2 + conf BCE (kept · unchanged)
2.35 · kept · Confusability-weighted location samplingCamera frame 128²×3 · ~1 m/px frozen contract C4 rotation fan 4 exact 90° turns 4×128²×3 64²×16 16²×24 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned GAP 48 layout 512 lighting gate brightness-blended pair of scorers σ probability field 32×32 cells, C4-vote averaged committed decode β-sharpened soft-argmax (unchanged) conf: field-shape hit predictor (lat, lon, confidence) frozen contract training-only — never flies fresh 6,000-place draw per bucket per epoch (kept · rd 20) 3 seeded relight realizations per bucket (kept · rd 17) 24 passes · cosine LR glide to zero (kept · rd 25) Gaussian-bump CE + coord L2 + conf BCE (kept · unchanged) NEW — hardest-impostor margin · λ 0.5 best-scoring cell outside the 7×7 ring around truth must trail the 3×3 truth block by ≥ 1.0 logit truth 3×3 hardest impostor
2.34 · reverted · Hardest-impostor margin lossCamera frame 128²×3 · ~1 m/px frozen contract C4 rotation fan 4 exact 90° turns 4×128²×3 64²×16 16²×24 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned GAP 48 layout 512 lighting gate brightness-blended pair of scorers σ probability field 32×32 cells, C4-vote averaged committed decode β-sharpened soft-argmax (unchanged) conf: field-shape hit predictor (lat, lon, confidence) frozen contract training-only — never flies crisp as today defocused NEW — per-crop sharpness roll ¼ pin-sharp (no resample) · ½ extra defocus σ∈U(0.4,1.4) px fresh 6,000-place draw per bucket per epoch (kept · rd 20) 3 seeded relight realizations per bucket (kept · rd 17) 24 passes · cosine LR glide to zero (kept · rd 25) Gaussian-bump CE + coord L2 + conf BCE
2.33 · reverted · Sensor-sharpness nuisance randomizationINFERENCE PATH — WHAT FLIES TRAINING-TIME CALIBRATION — NEVER FLIES 128²×3 camera frame one of 6 lighting renders frozen contract rotation fan four 90° turns MobileNetV3-Small trunk pretrained, 9 blocks 8²×48 crop descriptor 512-d layout + 48-d GAP probability field 32×32 cells · two experts · C4 vote β-sharpened decode — balance point of the hottest mode frozen contract (lat, lon, confidence) position fix + confidence field-shape statistics peak · entropy · decode gap self-trust score z − regime abstention bar (interpolated) confidence 0–1 conf < 0.3 → 'no fix' (must answer ≥20%) frame brightness six regime anchors abstention bar per regime, blended by frame brightness fenced calibration blocks ~10% of train blocks by stable hash — excluded from every training draw, same halo rule as eval blocks (grey); errors are measurable there, terrain is honestly unseen median error q* ≥ 0.26 keep rate per-regime operating point each lighting regime's keep rate picked at the low point of its fenced risk–coverage curve (floor 0.32) — the six thresholds become the anchor row above 2.32 · reverted · Risk-controlled abstentionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of 6 renders frozen contract rotation fan 4 turns, voted at the field 64²×16 8²×48 MobileNetV3-Small trunk pretrained, gently fine-tuned 4×4×8 crop fingerprint 1×1 conv + 2×2 pool · each tap = 32 m of ground bright/dark variants blended by the kept gate slide across the map 8 ch × ~217×~213 learned neural map 32 m per cell · lives in the weights match-score field ~45,000 positions · softmax 32 m pitch — was 217 m sharpened soft-argmax commit to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the map's shape — may abstain Gaussian-CE + decode L2 + conf BCE — unchanged same σ (≈330 m tolerance) in map units, graded on the ~45k-position field training data, sampler & 24-epoch cosine schedule unchanged
2.31 · reverted · Neural-map correlation fieldINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of 6 renders frozen contract C4 rotation fan 4 turns, logits voted 64²×16 8²×48 MobileNetV3-Small trunk ImageNet-pretrained, blocks 0–8 8×8 layout squeeze 1×1 conv → 512-d GAP 48-d 560-d 560-d descriptor 48 texture ⊕ 512 layout day specialist FC 560 → 1,024 each dusk specialist night specialist × w₁ × w₂ × w₃ lighting dispatcher softmax → w₁,w₂,w₃ crop brightness probability field 32×32 cells · C4 logit vote β-sharpened soft-argmax commits to the dominant mode frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing cross-entropy on the routing weights vs the TRUE lighting bucket buckets group as day / dusk / night — the labels are free at train time specialists' FC weights ride int8 in the exported ONNX export-only pack — ~7 m cost measured (exps 26/29); 3 full heads fit the 4 MiB gate Gaussian-CE + decode L2 + conf BCE — unchanged training data & schedule also unchanged (24 fresh-draw passes)
2.30 · reverted · Supervised lighting dispatcherINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of 6 lighting renders frozen contract C4 rotation fan 4 turns, logits averaged 64²×16 8²×48 16²×24 MobileNetV3-Small trunk ImageNet-pretrained blocks 0–8 — unchanged 1×1 conv 24 → 4 ch · flatten → 1,024-d GAP · 1×1 layout squeeze crop summary — 1,584-d 48-d GAP + 512-d 8×8 layout (as before) + 1,024-d stride-8 fine code — new night gate blends a 48-d dark scorer per crop (unchanged) FC 1,584 → 1,024 logits weight tables fly int8 — measured ±7 m probability field 32×32 cells · C4 logit vote (unchanged) β-sharpened soft-argmax commit to the dominant mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads its own field shape Gaussian-CE (σ=1.5 cells) + decode-L2 + conf-BCE — unchanged training procedure unchanged — 24 fresh-draw epochs, cosine LR → 0 2.29 · reverted · Stride-8 fine-layout tapCamera frame 128²×3, ~1 m/px frozen contract Rotation fan four 90° turns ×4 stem + early 64²×16 mid blocks 16²×24 blocks 4–8 8²×48 blocks 9–10 — reinstated dilated — 8²×96 layout code 608-d int8 FC store — reinstated quantized at export · ~7 m effect σ lighting gate Probability map 32×32 cells, C4 vote Decode sharpened balance point Confidence abstains on smeared maps (lat, lon, confidence) frozen contract abstain bar set on held-back ground — new ~10% of train blocks never enter a training draw; the bar is tuned on ground as unseen as the exam's training-only lane — losses and schedule unchanged: Gaussian-CE map target · committed-coord L2 · conf BCE · 24 fresh-draw epochs, cosine LR → 0 2.28 · reverted · Rerun, never scoredCamera frame 128²×3, ~1 m/px frozen contract Rotation fan four 90° turns ×4 stem + early 64²×16 mid blocks 16²×24 blocks 4–8 8²×48 blocks 9–10 — reinstated dilated — 8²×96 layout code 608-d int8 FC store — reinstated quantized at export · ~7 m effect σ lighting gate Probability map 32×32 cells, C4 vote Decode sharpened balance point Confidence abstains on smeared maps (lat, lon, confidence) frozen contract abstain bar set on held-back ground — new ~10% of train blocks never enter a training draw; the bar is tuned on ground as unseen as the exam's training-only lane — losses and schedule unchanged: Gaussian-CE map target · committed-coord L2 · conf BCE · 24 fresh-draw epochs, cosine LR → 0 2.27 · reverted · Unseen-ground confidence calibration unlocks the reinstated 3×-capacity trunkCamera frame 128²×3, ~1 m/px frozen contract Rotation fan four 90° turns ×4 stem + early 64²×16 mid blocks 16²×24 blocks 4–8 8²×48 blocks 9–10 — new dilated — 8²×96 layout code 608-d int8 FC store — new quantized at export · ~7 m effect σ lighting gate Probability map 32×32 cells, C4 vote Decode sharpened balance point Confidence abstains on smeared maps (lat, lon, confidence) frozen contract training-only lane — unchanged: Gaussian-CE map target · committed-coord L2 · conf BCE · 24 fresh-draw epochs, cosine LR → 0 2.26 · reverted · Deployment-envelope capacity scalingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre epoch 1 epoch 2 … epoch 24 a fresh draw of 6,000 locations per bucket, every epoch every epoch, training restarts on a freshly drawn set of places LR before: constant LR, hard stop at epoch 8 now: cosine glide to zero across a 3× longer run convergence-scaled training — 3× the steps, annealed to zero The fresh-draw sampler (kept, left) made memorizing crops useless — but the schedule still stops at 8 passes, loss still falling, LR never lowered. A probe shows pure underfit: train crops now localize no better than eval crops (~1 km both). This run trains 24 fresh-draw passes with the LR gliding down a cosine — the same sampler, finally trained to converge. 2.25 · kept · Convergence-scaled trainingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre epoch 1 epoch 2 … epoch 8 a fresh draw of 6,000 locations per bucket, every epoch every epoch, training restarts on a freshly drawn set of places kept from exp 20 — unchanged trunk feature grid 8²×48 identity turn · read before pooling 16² 32² 64² throwaway upsampling decoder 8²→64² · ~55k params · never exported 64²×3 redrawn daytime view from features alone 64²×3 daytime reference crop same spot & heading, free from sim L1 ×0.3 NEW — daytime-redraw auxiliary from its own features, the net must redraw this spot in clean daylight; all six renders share one daytime answer, so seeing through grain and darkness is the cheapest fit — decoder never flies 2.24 · reverted · Daytime-redraw auxiliary, rerunINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre epoch 1 epoch 2 … epoch 8 a fresh draw of 6,000 locations per bucket, every epoch every epoch, training restarts on a freshly drawn set of places kept from exp 20 — unchanged trunk feature grid 8²×48 identity turn · read before pooling 16² 32² 64² throwaway upsampling decoder 8²→64² · ~55k params · never exported 64²×3 redrawn daytime view from features alone 64²×3 daytime reference crop same spot & heading, free from sim L1 ×0.3 NEW — daytime-redraw auxiliary from its own features, the net must redraw this spot in clean daylight; all six renders share one daytime answer, so seeing through grain and darkness is the cheapest fit — decoder never flies
2.23 · reverted · Daytime-redraw auxiliaryINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre epoch 1 epoch 2 … epoch 8 a fresh draw of 6,000 locations per bucket, every epoch every epoch, training restarts on a freshly drawn set of places kept from exp 20 — unchanged trunk feature grid 8²×48 identity turn · read before pooling 64 per-patch map fields — 3 of 64 shown one shared 1×1 conv 48→1024 · training-only, deleted before export NEW — per-patch place supervision every feature cell must place ITS OWN 16 m patch of ground on the map, graded by a Gaussian bump at that patch's true spot = crop centre + rotated cell offset. 64 graded answers per crop instead of 1. The flying network is untouched — the aux head never ships. 2.22 · reverted · Train-only per-patch place supervisionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre epoch 1 epoch 2 … epoch 8 a fresh draw of 6,000 locations per bucket, every epoch every epoch, training restarts on a freshly drawn set of places per-epoch location resampling — same training budget, 8× the places seen the last kept model studied the SAME 36,000 crops for all 8 epochs — same places, same headings, same simulator rolls — so memorizing each crop was the easiest way down the training loss. Now every epoch redraws its 6,000 locations per bucket (fresh headings and render mix): ~30k of the 45k train places get seen, each only ~1–2×. Memorizing single crops stops paying; only cues that transfer between places do — the skill an eval crop over never-trained terrain actually demands. 2.20 · kept · Per-epoch training-set resamplingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 · ~1 m/px camera frame one of six lighting renders frozen contract shared weights ×4 rotation fan 0°·90°·180°·270° — exact pixel turns MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 1024 logits ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map’s shape (unchanged) may abstain position + confidence losses — all unchanged Gaussian-CE + decode L2 + conf BCE — still graded at the true centre half of each bucket’s training crops are patched before batching crop from elsewhere same lighting bucket lift a 48–96 px block patched training crop label: still the true centre off-site distractor patching — p = 0.5, training only eval crops sit on never-trained blocks: the centre terrain is always unfamiliar, familiar terrain pokes in from an edge (~76% of eval crops contain some trained pixels). The probe behind exp 18: train-split crops localize to 267–400 m, eval-split to 708–1153 m — a ~3× structural gap, in every bucket. Training crops were always 100% familiar, so the whole-crop head never practised partial recognition; patched crops force it to answer from the content consistent with one place and down-weight the rest. 2.19 · reverted · Off-site distractor patchingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of six lighting renders frozen contract rotation fan 0°·90°·180°·270° shared weights ×4 MobileNetV3-Small trunk ImageNet init (BSD-3) 8²×48 ×4 patch voters every cell = one voter 64 per-patch field votes shared 1×1 MLP 48→128→1024 — one field per patch ⋯ ×64 votes, each × learned weight wᵖ (softmax over patches) voted probability field weighted blend of 64 patch votes mean over 4 turns (unchanged) β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map's shape (unchanged) may abstain dense per-patch supervision — 64 graded answers per crop each patch's vote is graded against a Gaussian bump on the map cell of ITS OWN ground content, computed exactly at train time as crop center + R(heading)·patch offset — a place can no longer be recognized by one whole-crop gestalt: all 64 voters must independently know where their ground sits probe, midday, matched headings: train-crop median 267–400 m vs held-out 708–1153 m — the global template head memorizes but does not generalize fused-field Gaussian-CE + decode L2 + conf BCE — unchanged · three-realization training renders (exp 17) — unchanged · calibration unchanged 2.18 · reverted · Dense per-patch field votingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of six lighting renders frozen contract rotation fan 0°·90°·180°·270° — exact pixel turns shared weights ×4 MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map's shape (unchanged) may abstain Gaussian-CE + decode L2 + conf BCE — all unchanged the losses now grade the voted answer — train what flies seed₀ = the eval render seed₁ fresh dice roll seed₂ fresh dice roll each training crop is drawn from one of three seeded realizations nuisance-randomized training renders the frozen relight sim rolls dice for sensor noise + lamp thinning; the stored night render freezes ONE roll, ≈ half that image's content (mean |Δpx| 29/255 between rolls vs 6/255 at midday; render mean 60/255) → the 1024-way head memorizes roll-specific texture that cannot transfer to held-out locations. Training now draws each bucket's crops from three seeded rolls (⅓ each), so only the seed-stable structure — roads, buildings, where the lit areas sit — stays discriminative between locations. eval renders, model, losses, decode, calibration: all unchanged — train-time only 2.17 · kept · Nuisance-randomized training rendersINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one of six lighting renders frozen contract rotation fan 0°·90°·180°·270° — exact pixel turns shared weights ×4 MobileNetV3-Small trunk ImageNet init (BSD-3) · gently fine-tuned 8²×48 ×4 four field maps same gated two-expert head reads each turn 32×32 cells voted probability field mean of four logit maps = geometric-mean vote β-sharpened soft-argmax commits to the hottest mode (unchanged) frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 reads the voted map's shape (unchanged) may abstain invariance by construction, not by data exp 6 taught headings by example — reverted; this builds them in Gaussian-CE + decode L2 + conf BCE — all unchanged the losses now grade the voted answer — train what flies 2.16 · kept · C4 rotation-vote fieldINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY red = this experiment: field-shape confidence + calibrated abstention — unsure crops return no fix 128²×3 camera frame one night exposure frozen contract MobileNetV3-Small trunk ImageNet-pretrained · stem + 8 inverted-residual blocks, unchanged 128²×3 8²×48 layout code 1×1 conv, unchanged 8²×8 48-d GAP unchanged ∑/n raw-pixel mean brightness lighting gate tiny MLP → σ, unchanged g ∈ [0,1] bright head FC 560→1024 dark head FC 48→1024 · GAP-only, unchanged blend (1−g)·b + g·d probability field 32×32 cells β·logits sharpened field softmax(β·logits) β = 3 — unchanged soft-argmax over the sharpened field commits to the dominant peak — unchanged frozen contract (lat, lon, confidence) position fix + confidence Gaussian-CE on the field + L2 on the committed decode both unchanged — localization trains exactly as before peak mass entropy mode−mean gap 0.3 confidence head — learns when to abstain tiny MLP on field shape · calibrated to the frozen 0.3 bar below the bar → no fix, coast on last estimate confidence BCE — did the committed fix land within ~350 m? detached head — cannot disturb localization per-bucket threshold calibration ≥40% keep a fix in every lighting bucket 2.15 · kept · Selective predictionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY red = this experiment: β-sharpened peak-commit decode replaces mean-of-map averaging 128²×3 camera frame one night exposure frozen contract MobileNetV3-Small trunk ImageNet-pretrained · stem + 8 inverted-residual blocks, unchanged 128²×3 8²×48 layout code 1×1 conv, unchanged 8²×8 48-d GAP unchanged ∑/n raw-pixel mean brightness lighting gate tiny MLP → σ, unchanged g ∈ [0,1] bright head FC 560→1024 dark head FC 48→1024 · GAP-only, unchanged blend (1−g)·b + g·d probability field 32×32 cells β·logits sharpened field — NEW softmax(β·logits), β = 3 peaks boosted, hedging suppressed soft-argmax over the sharpened field commits to the dominant peak frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing Gaussian-CE on the unsharpened field (unchanged) the field keeps learning calibrated evidence L2 on the SHARPENED decode the loss now grades the committed answer that flies 2.14 · kept · Peak-commit decodeINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract MobileNetV3-Small trunk ImageNet-pretrained · stem + 8 inverted-residual blocks (exp 11, unchanged) 128²×3 8²×48 layout code 1×1 conv, unchanged 8²×8 48-d GAP unchanged ∑/n raw-pixel mean brightness lighting gate tiny MLP → sigmoid g ∈ [0,1] bright head FC 560→1024 dark head — NEW FC 48→1024, GAP-only blend (1−g)b + g·d probability field 32×32 cells, unchanged soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing cross-entropy + decode L2 (unchanged) gate + dark head train on the SAME losses no lighting label — g learns from brightness ⊕ GAP red = this experiment: gated dark-expert head blended with the existing layout head 2.12 · kept · Luminance-gated dark-expert field head blended with the existing layout headINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract MobileNetV3-Small encoder — ImageNet-pretrained stem + 8 inverted-residual blocks · depthwise conv + squeeze-excite 64²×16 8²×48 1×1 conv → 8²×8 layout code keeps what sits WHERE in the crop 48-d GAP texture average — as before 512 ⊕ 48 = 560 FC 560 → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing trunk weights arrive from ImageNet (1.2M real photos) — not random fine-tuned gently: trunk lr 1e-4 · fresh heads lr 1e-3 · BSD-3 licensed Gaussian-CE + decode L2 + conf BCE — all unchanged head, decode and training recipe identical to kept exp 10 red = this experiment: an ImageNet-pretrained trunk replaces the from-scratch encoder 2.11 · kept · ImageNet-pretrained MobileNetV3-Small trunk replaces the from-scratch encoderINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 1×1 conv → 8²×8 layout code keeps what sits WHERE in the crop 128-d GAP texture average — as before 512 ⊕ 128 = 640 FC 640 → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing Gaussian-CE + decode L2 + conf BCE — all unchanged training recipe identical to kept exp 7; only the head's input changed red = this experiment: the field head reads an 8×8 layout code ⊕ texture average 2.10 · kept · Layout-aware field headINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing = same place training batches become PAIRS — same place, two lighting renders, two headings one shared 6,000-location set across all 6 renders · 32 places × 2 views per step NT-Xent: pull the pair together, push 62 others apart 64-d projection of the descriptor · τ 0.2 · head never flies Gaussian-CE + L2 + conf BCE — unchanged 2.9 · reverted · Cross-lighting contrastive pairsINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract + + + residual encoder — 11 conv layers stem + 3 stride-2 stages, each ending in a residual block ~973k params, 4.2× the old 4-conv stack 3.7 of the 4.0 MiB flight-memory budget (was 0.9) 64²×32 8²×160 160-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing Gaussian-CE + L2 + conf BCE (unchanged) same lessons, same 36k crops, same 8 epochs — only the student grew 2.8 · reverted · Deployment-envelope residual encoderCamera frame 1 of 6 lighting renders 128×128×3 64²×16 32²×32 16²×64 8²×128 Feature extractor 4 strided conv blocks · ~230k params 128-d embedding global average pool 128 → 1024 probability field 32×32 map cells · ~220 m/cell soft-argmax expected coordinate over cell centers (lat, lon, confidence) position + confidence u,v ∈ [0,1] map coords → lat, lon conf head (linear + sigmoid) training only was: 800 places/bucket now: 6,000 places/bucket of ~45,000 available ×7.5 coverage — every map cell now trained in every lighting bucket; ~4,500 steps (was 600) Gaussian-CE on smoothed cell target + coord L2 + conf BCE (unchanged) one static crop per place at one random heading (unchanged, per exp-6) 2.7 · kept · Scale training coverage 7.5xINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing crop rotations re-drawn EVERY epoch fresh views of the same places each pass, not one frozen tensor reused 8× Gaussian-CE + L2 (unchanged) 2.6 · reverted · Per-epoch rotation resampling replaces the one-shot frozen training tensorINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing sum-pool the SAME field to 16², 8², 4² — Gaussian-CE at every scale coarse cells collect 10–40 positives each and steer the fine field (+ L2 on decode, unchanged) 2.5 · reverted · Hierarchical coarse-to-fine supervision of the probability field via probability poolingINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 dilated conv 3×3, d2 ~95 px context per cell 64 per-patch coordinates 1×1 conv + σ (8×8×2) mean of 64 answers one committee answer from 64 votes frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing smooth-L1: each patch vs its OWN true coordinate crop center + rotated offset → 64× denser supervision 2.4 · reverted · ACE-style dense per-patch scene-coordinate regression replaces global map-cell probability fieldINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map argmax → 5×5 window soft-argmax commit to the hottest cell, refine locally frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing cross-entropy vs Gaussian cell target (unchanged) L2 on the WINDOWED decode train on exactly what flies 2.3 · reverted · Argmax-anchored local soft-argmax decode replaces global expected-coordinateINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d GAP FC → 1024 logits softmax probability field 32×32 cells over the map soft-argmax over ALL cells answer = Σ probability · cell-center frozen contract (lat, lon, confidence) position fix + confidence confidence 0–1 may abstain instead of guessing cross-entropy: match a Gaussian bump on the true cell σ = 1.5 cells — every cell gets a hotter/colder gradient L2 on the decoded (u, v) 2.2 · kept · DSNT-style spatial probability field over the map replaces direct (u,v) regressionINFERENCE PATH — WHAT FLIES TRAINING SIGNALS — NEVER FLY 128²×3 camera frame one night exposure frozen contract convolutional encoder Conv 3×3, stride 2 ×4 · BN + ReLU 64²×16 8²×128 128-d vector global avg pool FC 128 → 3 sigmoid one shot frozen contract (lat, lon, confidence) position fix + confidence mean-squared error on (u, v) the whole map supervises one number pair BCE ×0.1 on confidence starting design — nothing changed yet 2.1 · kept · Starting baseline — naive TinyLocNet, from-scratch, frozen pipeline v1
Our training data
Everything the model knows, it learned from three open-licensed
aerial photographs of the same Berlin bounding box, each fetched by the
frozen pipeline onto one shared grid at 1 m per pixel
(6,939 × 6,828 pixels): the Brandenburg DOP20 orthophoto
(© GeoBasis-DE/LGB, dl‑de/by‑2‑0, modified)
and Berlin’s TrueDOP surveys of 2022 (leaf-off) and 2023 (early
spring) (Geoportal Berlin, dl‑de/zero‑2‑0). It is scored
on a fourth, TrueDOP 2024 , which never enters training in any form:
held-out frames stand on the same ground, framed from positions and
headings training never used, in a photograph taken in a different year.
No Google or Bing tiles anywhere: only open geodata, so the entire project
can be published. A hundred of the 2,970 tiles:
52.52829 · 13.32061 52.52845 · 13.33192 52.52858 · 13.34135 52.52874 · 13.35267 52.52890 · 13.36398 52.52903 · 13.37342 52.52918 · 13.38473 52.52934 · 13.39605 52.52946 · 13.40548 52.52962 · 13.41680 52.52139 · 13.32087 52.52155 · 13.33218 52.52168 · 13.34161 52.52184 · 13.35293 52.52200 · 13.36424 52.52213 · 13.37367 52.52228 · 13.38498 52.52243 · 13.39630 52.52256 · 13.40573 52.52271 · 13.41704 52.51564 · 13.32109 52.51580 · 13.33240 52.51593 · 13.34183 52.51609 · 13.35314 52.51624 · 13.36445 52.51637 · 13.37388 52.51653 · 13.38520 52.51668 · 13.39651 52.51681 · 13.40594 52.51696 · 13.41725 52.50874 · 13.32135 52.50890 · 13.33266 52.50903 · 13.34209 52.50919 · 13.35340 52.50934 · 13.36471 52.50947 · 13.37414 52.50963 · 13.38545 52.50978 · 13.39676 52.50991 · 13.40619 52.51006 · 13.41750 52.50298 · 13.32157 52.50314 · 13.33288 52.50328 · 13.34231 52.50343 · 13.35361 52.50359 · 13.36492 52.50372 · 13.37435 52.50388 · 13.38566 52.50403 · 13.39697 52.50416 · 13.40639 52.50431 · 13.41771 52.49608 · 13.32183 52.49624 · 13.33314 52.49637 · 13.34256 52.49653 · 13.35387 52.49669 · 13.36518 52.49682 · 13.37460 52.49697 · 13.38591 52.49713 · 13.39722 52.49726 · 13.40664 52.49741 · 13.41795 52.49033 · 13.32205 52.49049 · 13.33336 52.49062 · 13.34278 52.49078 · 13.35409 52.49094 · 13.36539 52.49107 · 13.37482 52.49122 · 13.38612 52.49138 · 13.39743 52.49150 · 13.40685 52.49166 · 13.41816 52.48343 · 13.32232 52.48359 · 13.33362 52.48372 · 13.34304 52.48388 · 13.35435 52.48404 · 13.36565 52.48417 · 13.37507 52.48432 · 13.38638 52.48447 · 13.39768 52.48460 · 13.40710 52.48475 · 13.41841 52.47768 · 13.32253 52.47784 · 13.33384 52.47797 · 13.34326 52.47813 · 13.35456 52.47829 · 13.36586 52.47841 · 13.37528 52.47857 · 13.38659 52.47872 · 13.39789 52.47885 · 13.40731 52.47900 · 13.41861 52.47078 · 13.32280 52.47094 · 13.33410 52.47107 · 13.34352 52.47123 · 13.35482 52.47138 · 13.36612 52.47151 · 13.37554 52.47167 · 13.38684 52.47182 · 13.39814 52.47195 · 13.40756 52.47210 · 13.41886
5
How to use it One file, one function, three steps.
1. Download the model: berlin.onnx (31.7 MB). It runs anywhere ONNX
runs — a laptop, a Raspberry Pi. The ESP32‑P4 needs it
converted to 8-bit integers first, which has not been done yet.
2. Feed it camera frames, 3. trust it only when it is sure —
the whole client is this:
# pip install onnxruntime opencv-python pyproj numpy
import cv2, numpy as np, onnxruntime as ort
from pyproj import Transformer
# the Berlin model's georeference (from the training raster's metadata)
ORIGIN = (385883.0, 5821316.0) # UTM 33N of the map's top-left corner
SIZE = (6939, 6828) # map size in px; 1 px = 1 m
to_wgs84 = Transformer.from_crs("EPSG:32633", "EPSG:4326")
sess = ort.InferenceSession("berlin.onnx")
cam = cv2.VideoCapture(0) # the downward-facing camera
ok, bgr = cam.read() # 1 - grab one frame
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
rgb = cv2.resize(rgb, (128, 128)) # frame should cover ~128 m of ground
x = rgb.astype(np.float32).transpose(2, 0, 1)[None] / 255.0
u, v, conf = sess.run(None, {"frame": x})[0][0] # 2 - ask the network
lat, lon = to_wgs84.transform(ORIGIN[0] + u * SIZE[0],
ORIGIN[1] - v * SIZE[1])
if conf < 0.3: # 3 - trust it only when it is sure
print("no fix - the model abstains")
else:
print(f"{lat:.5f}, {lon:.5f} (confidence {conf:.2f})")
The frame must look straight down over Berlin and
cover roughly 128 m of ground (about 100 m altitude with a
typical lens). Below confidence 0.3 the fix is discarded, exactly as the
research scores it.
6
Explore the record Every experiment, every design, every dead end.
The research log
Every experiment ever run, failures included: pre-registered
hypotheses, results, per-area × lighting scoreboards, the exact prompts
the agents received, and one real worked example per experiment — the same
held-out viewpoint through each model's actual deployed weights.
Experiment
lineage
The family tree of the search: every experiment as one node in
discovery order, arcs to the design it built on — hover any node to trace
its ancestry, kept trunk and dead branches alike.
Flight path
The model flying a simulated UAV over the 2024 photograph: 100
random flights from a to b, then six other missions — return home,
a tour, a loiter, a survey grid, a cold start and the Spree — each
replayable, failures included.
Model
designs
The technical figures: each experiment's model design, drawn by the
agent itself before training, in one shared visual language — frozen
endpoints aligned so you can scroll and compare designs directly.
7
Proven alternatives GPS-denied localisation is not unsolved — this just walks a different road.
GPS-denied visual localization is not an unsolved problem. The
established, field-tested family matches live camera frames against
georeferenced reference imagery carried on the aircraft — e.g.
WildNav
(Vision-based GNSS-Free
Localization for UAVs in the Wild ), which matches UAV photographs
against satellite tiles with deep feature matching and demonstrated
GNSS-comparable accuracy in real flights. If you need working GPS-denied
navigation today, start there, not here.
This project deliberately walks a different road, for two reasons.
Licensing: the strongest matchers in that stack — Magic Leap's
SuperGlue
/ SuperPoint
pretrained networks — are licensed for noncommercial research only, without
the right to sublicense, which is incompatible with a fully open-sourceable,
commercially usable system (permissive alternatives like
LightGlue (Apache-2.0)
exist, but the whole approach still means shipping reference imagery on
the airframe — this project's hardest constraint rules that out).
Curiosity: the actual motivation is ground-level research — can an
autonomous loop of coding agents discover a genuinely different approach,
with no reference imagery on board and the map living entirely in the
network's weights, under a low-light premise? It may or may not end up
competitive with the field-tested systems above. Saying so out loud is
part of the experiment.
The vision, if it works: a novel algorithm that lets anyone draw
a bounding box around a place they care about and generate their own tiny,
punchy, self-contained geo-boxed model — reliable, personalized, and free.
Open data in, open weights out; no reference imagery to license, no vendor
to call, no cloud to depend on. The map is yours, and it lives in a few
megabytes you own.
updated 2026-11-04 12:37 UTC · experiments run on a single local
machine; the loop commits every result to git as it goes