A startup says its AI beats Claude and GPT-5.5 at reproducing research — on its own numbers, for now· 4h ago

Hand an AI a research paper, have it turn the method into working code, and see whether it can reproduce the reported results — this is one of the harder ways to measure a model. It demands not recall but comprehension, implementation and the repair of failures, all in one chain. A startup founded by ex-DeepMind researchers says it has topped both Anthropic's Claude and OpenAI's GPT-5.5 on exactly this paper-reproduction benchmark.
But this is the company's own announcement. Who chose which papers, and what counted as a successful reproduction? The numbers move with the design of the test. A claim to have passed the frontier carries weight only once outsiders reproduce it under the same conditions. This paper has spent the week watching benchmark scores travel far ahead of any verification, and today's claim starts in that same holding pen.
What matters here is the direction, not the figure. The industry assumes the frontier is the private property of a few heavily capitalized firms. That a small team — however pedigreed — can even claim a place among them suggests the top of the table may not be fixed. Verification will decide the rest.