EVERY RUN, INCLUDING THE BAD ONES
We ran it twenty-eight times and published all of it.
One pass over five fragments is a sample, not a measurement. We found that out the hard way: our first published result recorded zero false-confident identifications, and repeating it showed that figure was one lucky pass.
This page is the whole record, rendered from the same JSON the tests assert against. Nothing on it has been selected to flatter the system.
Across every recorded run
Every run, case by case
Five fragments from the Library of Congress, each with a sealed answer key. A run is one complete investigation. Nothing here is averaged or omitted.
Loading the study...
What changed when we fixed the defects we found
The study exposed two: candidates that were not films at all, and hypotheses that nothing ever competed against. Both were fixed in code. Only one of them improved the results, and the other outcome is the more interesting one.
Measured against the alternative, not against nothing
Before measuring this system we measured what a capable model does without it: same five fragments, same model, no web access and no gate.