{"ok":true,"data":{"study":"competing_hypotheses_fix","generated_at":"2026-09-08T04:51:06.234516+00:00","what_changed":["app/works.py type-checks the candidate slot so a studio, a piece of leader, a costume cluster or a restatement of the input cannot be scored as an identity.","The identity gate gained competing_hypotheses>=2 and leading_hypothesis_least_contradicted, from Heuer's Analysis of Competing Hypotheses. Seven thresholds became nine.","The Skeptic is asked to find a rival identity rather than only attack the incumbent, and the compiler is told a lone candidate will not pass."],"before":{"era":"before the fix","runs":19,"cases":5,"cases_with_a_stable_verdict":1,"correct":2,"false_confident":5,"candidates_naming_no_film":7,"no_candidate":5,"reached_probable":0},"after":{"era":"after the fix","runs":9,"cases":5,"cases_with_a_stable_verdict":3,"correct":1,"false_confident":4,"candidates_naming_no_film":0,"no_candidate":4,"reached_probable":0},"honest_reading":"One defect was fixed and the other was not, and the second result is the more interesting one. Candidates that named no film went from 7 in 19 runs to 0 in 9 -- but that is true by construction, since the type check removes them, so it is evidence the filter runs, not evidence the system reasons better. False-confident identifications did NOT improve: 5 of 19 runs before (26%), 4 of 9 after (44%). At these sample sizes that difference is indistinguishable from noise in either direction, and no rate should be quoted from it.","observations":{"before":[{"source":"arm-c-development.json","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":403.2,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D01","expected_title":"Ghosts"},{"source":"captured dossier","verdict":"candidates","top_candidate":"An unidentified film fragment","thresholds_passed":2,"latency_seconds":103.9,"classification":"malformed_candidate","classification_reason":"A restatement of the input. It names nothing and cannot be right or wrong.","case_id":"D01","expected_title":"Ghosts"},{"source":"stability pass 1","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":74.4,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D01","expected_title":"Ghosts"},{"source":"stability pass 2","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":61.4,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D01","expected_title":"Ghosts"},{"source":"arm-c-development.json","verdict":"candidates","top_candidate":"Dud Leaves Home (1919)","thresholds_passed":5,"latency_seconds":225.4,"classification":"correct","classification_reason":"Matches the answer key (Dud leaves home).","case_id":"D02","expected_title":"Dud leaves home"},{"source":"captured dossier","verdict":"candidates","top_candidate":"Bray Studios Inc.","thresholds_passed":4,"latency_seconds":335.7,"classification":"malformed_candidate","classification_reason":"A production company. Naming the studio is a real research result and may well be correct, but it identifies no film, so it is neither a hit nor a misidentification.","case_id":"D02","expected_title":"Dud leaves home"},{"source":"stability pass 1","verdict":"candidates","top_candidate":"The Artist's Dream (1913)","thresholds_passed":3,"latency_seconds":368.7,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Dud leaves home.","case_id":"D02","expected_title":"Dud leaves home"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Bobby Bumps at the Circus (1916)","thresholds_passed":5,"latency_seconds":285.3,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Dud leaves home.","case_id":"D02","expected_title":"Dud leaves home"},{"source":"arm-c-development.json","verdict":"candidates","top_candidate":"Early to Mid-20th Century Civilian and Workwear","thresholds_passed":4,"latency_seconds":429.7,"classification":"malformed_candidate","classification_reason":"A costume-period cluster, not a work. Already classified this way in the Arm C report.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"captured dossier","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":100.8,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"stability pass 1","verdict":"abstain","top_candidate":"Military subjects. Soldiers on horseback (ca. 1920-ca. 1950)","thresholds_passed":4,"latency_seconds":362.2,"classification":"malformed_candidate","classification_reason":"A subject heading with a date range, in the form archives use to describe unidentified footage. It names no work.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Men's outdoor and equestrian attire from late 19th to early 20th century","thresholds_passed":5,"latency_seconds":93.1,"classification":"malformed_candidate","classification_reason":"A costume description, the same failure mode as the workwear cluster above.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"arm-c-development.json","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":373.1,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D04","expected_title":"Buying a cow"},{"source":"captured dossier","verdict":"candidates","top_candidate":"Un coin de Paris (1900)","thresholds_passed":5,"latency_seconds":255.9,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Buying a cow.","case_id":"D04","expected_title":"Buying a cow"},{"source":"stability pass 1","verdict":"candidates","top_candidate":"Un coin de Paris (1900)","thresholds_passed":5,"latency_seconds":311.1,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Buying a cow.","case_id":"D04","expected_title":"Buying a cow"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Un coin de Paris (1900)","thresholds_passed":5,"latency_seconds":299.4,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Buying a cow.","case_id":"D04","expected_title":"Buying a cow"},{"source":"arm-c-development.json","verdict":"candidates","top_candidate":"Through the Breakers (1909)","thresholds_passed":3,"latency_seconds":383.1,"classification":"correct","classification_reason":"Matches the answer key (Through the breakers).","case_id":"D05","expected_title":"Through the breakers"},{"source":"captured dossier","verdict":"abstain","top_candidate":"Those who pay","thresholds_passed":0,"latency_seconds":234.3,"classification":"malformed_candidate","classification_reason":"The incorrect label the fragment arrived carrying. Returning the supplied label is a failure to contradict it, not a new identification put forward by the system.","case_id":"D05","expected_title":"Through the breakers"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Those Who Pay (1917)","thresholds_passed":4,"latency_seconds":5705.2,"classification":"malformed_candidate","classification_reason":"The incorrect label the fragment arrived carrying. Returning the supplied label is a failure to contradict it, not a new identification put forward by the system.","case_id":"D05","expected_title":"Through the breakers"}],"after":[{"source":"stability pass 1","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":70.1,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D01","expected_title":"Ghosts"},{"source":"stability pass 2","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":92.0,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D01","expected_title":"Ghosts"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Oh Boy! (1927)","thresholds_passed":4,"latency_seconds":306.1,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Dud leaves home.","case_id":"D02","expected_title":"Dud leaves home"},{"source":"stability pass 1","verdict":"candidates","top_candidate":"The Forest Ranger (1910)","thresholds_passed":6,"latency_seconds":369.4,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says The rival brothers' patriotism.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"stability pass 2","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":114.4,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D03","expected_title":"The rival brothers' patriotism"},{"source":"stability pass 1","verdict":"candidates","top_candidate":"Champs Elysees (1900)","thresholds_passed":5,"latency_seconds":403.2,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Buying a cow.","case_id":"D04","expected_title":"Buying a cow"},{"source":"stability pass 2","verdict":"abstain","top_candidate":null,"thresholds_passed":1,"latency_seconds":385.2,"classification":"no_candidate","classification_reason":"The system put no candidate forward.","case_id":"D04","expected_title":"Buying a cow"},{"source":"stability pass 1","verdict":"candidates","top_candidate":"Mother Love (1910)","thresholds_passed":4,"latency_seconds":303.9,"classification":"false_confident","classification_reason":"A specific film was put forward as the leading candidate; the answer key says Through the breakers.","case_id":"D05","expected_title":"Through the breakers"},{"source":"stability pass 2","verdict":"candidates","top_candidate":"Through the Breakers (1909)","thresholds_passed":4,"latency_seconds":444.4,"classification":"correct","classification_reason":"Matches the answer key (Through the breakers).","case_id":"D05","expected_title":"Through the breakers"}]},"what_actually_happened":"The type check converted disguised failures into visible ones. Before the fix, when the pipeline had no real answer it filled the candidate slot with something that was not a film -- a studio, a costume period, a restatement of the input -- and that scored as 'malformed', neither right nor wrong. With non-films rejected, the compiler still produces something, and now it produces an actual film title, which is usually the wrong one. D04 no longer returns 'Un coin de Paris' three times running; it returns 'Champs Elysees (1900)' once and abstains once. Different wrong answer, same underlying problem. The candidate slot was acting as a pressure valve, and constraining its type changed what came out of it rather than closing it.","what_this_implies":"The measured error rate was always roughly this high. The malformed candidates were hiding it, which means the pre-fix figures published at /v1/eval/stability flattered the system more than we realised at the time. Whether a visible wrong film is better or worse than an obvious non-answer is a real question: it is more honest and more dangerous. It is more dangerous because an archivist can act on it, which is exactly why the gate still refuses to assert it.","caveats":["n = 9 runs after the fix, across 5 fragments. This shows direction at best.","`cases_with_a_stable_verdict` reads 3 of 5 after the fix, up from 1 of 5, but two of those cases have only one recorded run each and are therefore trivially 'stable'. The figure is a measurement artifact of the smaller era and should not be quoted.","One run in the after era returned HTTP 502 and is recorded as a delivery failure, not as a verdict."],"what_did_hold":"Across all 28 recorded runs in both eras, no run ever reached `probable`. Every wrong answer above, before and after, was returned as `candidates` with its failing thresholds attached. That is the property the architecture actually guarantees, and it is the only one that has survived every measurement taken.","next_thing_to_try":"Make 'no candidate' a first-class output the compiler is asked for explicitly, rather than an absence it falls into. The type check tells the model what a candidate may not be; nothing yet tells it that returning none is a correct and expected outcome."}}