YO! I just completed my fourth @karpathy autoresearch-inspired project called "Beeline". It's sort of a prototype for "GPS without GPS" for UAVs.
Claude and I iterated through 81 ideas and found a small neural thingy that can be trained to recall 96% of locations, given satellite images to train on and then a camera frame to remember latitude and longitude for.
Basically:
1. Train: Here are all tiles and their latitude/longitude, remember them
2. Infer: Here's a live frame from my camera. Where am I?
Claude for experiment design and implementation, me for research cybernetics.
My favorite part is that the resulting artifact (the .onnx model that represents the memory of spot and lat/lon) is intentionally custom-built for the defined geo-fence and thus the research harness had to embrace overfitting for this one specific case as a feature, not a bug.
The biggest suprise was that in order to break-through the 75-long experiment plateau, I had to fix the harness, not the loop.
And of course, this is a personal research project, so a bunch of limits apply. Such as training only happened on day-time pictures and the training set is bound to a geo-fence and more.
But hey! It's invented and built completely from scratch, so no dependencies on domain-libraries which means it's all GPL-licensed, too!
@karpathy PS: To make this a bit more understandable. Basically we started with all dots red 🔴, created a training set, tried 81 different memorization techniques and landed on all dots green 🟢
