THE DEMO SET
Ten fragments, chosen so that five of them should not be identified.
Every one is a 50-second excerpt from the Library of Congress National Screening Room, published by 1929 and in the United States public domain. Watch them, download them, and run the five development cases yourself.
A demo set of ten easy wins would prove nothing. This one is balanced by required behaviour: two cases must be abstained on, two must return ranked candidates rather than a verdict, and one arrives with a catalogue label that the record itself contradicts. A system that identifies all ten has failed.
DEVELOPMENT SPLIT · RUNNABLE
Five you can run right now.
These are the cases used to build the system, so their behaviour is known. Running one calls the public API with a key this page mints for itself.
HELD-OUT SPLIT · SEALED
Five that stay closed.
A benchmark you can re-run until it passes measures nothing. These five are listed with their SHA-256 hashes so the set cannot be quietly changed, and the media endpoint refuses to serve them. Try it: it returns 423. They are opened exactly once, after the implementation is tagged eval-freeze-*, and every miss is published alongside the successes.
RIGHTS AND PROVENANCE
Where these came from, and what was done to them.
Each fragment is five adjacent ten-second segments taken from one Library of Congress item, joined, stripped of container metadata and chapters, transcoded to H.264, and silenced. One source carried an identifying footer baked into the scan; the builder cropped it and the transform is recorded on that case. Offsets were chosen only after contact-sheet review, so no case was selected by peeking at an answer.
Required credit, and it appears on every response and download: Library of Congress, Motion Picture, Broadcasting, and Recorded Sound Division. The Library states it is unaware of U.S. restrictions for the vast majority of these films but places item-specific copyright, privacy, publicity and trademark assessment on the user.