Science Machine · PCOS, audited

What the machine gets wrong

Measured failures first, then the rules our own agents broke.

1. It rarely notices that a question has already been answered

We gave the machine four questions whose answers were already in print before a cut-off year. It should find the answering paper and close the gap, or narrow it with that paper named. bench/SCORECARD.md:7

ReadingResult
Headline: every gap from the papers that raised the question was closed, or narrowed with the answering paper named1 of 4
Strict: every matching gap, including neighbouring questions0 of 4
Best matching gap closed or narrowed on the answering paper4 of 4
Latest round, 2026-11-07. bench/SCORECARD.md:11-15

Across the four questions, 9 of 20 matching gaps were still left open when the answer was in print. palace/threads/challenge-closes-nothing.md:24 In the PCOS sweep, the search for prior work closed 1 of 123 gaps. palace/threads/challenge-closes-nothing.md:37

So the machine can find the paper that answers a question, but the gaps next to it often stay open. Any gap it reports may already be answered. That is why every gap carries the searches that were run and the closest work found.

2. The same question can come out differently on different runs

One test question went closed, narrowed, closed, narrowed across four versions of the code. bench/SCORECARD.md:45-49 docs/aar/AAR_2026-11-07_r-2026-11-07-1.md:29-30

3. "Nobody has studied this" is usually too strong

In a fact-check of an AI's claims about sickle cell trait, every "nobody has studied X" claim was softer than stated: 6 of 6. palace/threads/challenge-closes-nothing.md:38

4. Readers miss things, and two copies of one model miss the same things

5. Code errors found by review

6. Facts supplied from memory

This is why the rule is that a model never supplies an ID, a year or a count. Code looks them up.

7. Rules our agents broke

Every rule that lived only in an agent's instructions has been broken at least once. palace/threads/guardrails.md:7

The fix is to move rules out of instructions and into code that refuses the action, then test that the refusal works.

8. Cost estimates run over