Glaneur
Glaneur AI

NOV 8 - Nine AI agents fail an open research test

Sunday, November 8th · 2 min 30

In this episode

Full transcript

Good morning. It's Sunday, November 8th, and here's the AI news, from France.

We start with a sobering test for AI in science. Researchers led by Carnegie Mellon University, working with US national labs, asked nine AI agents to tackle open research questions. None of the 135 submissions met the experts' bar. In 114 cases, the papers failed a basic integrity check. Most claimed work that was never done, or cited numbers that contradicted their own logs. The authors say the benchmark is meant to improve research agents, not to rule them out.

Staying with models, the American startup Nous Research has released Arete-27B. It is a fine-tuned version of Alibaba's open model Qwen3.8. Training on 53 million tokens cut the rate of Chinese state propaganda in its answers from 22 percent to under one percent. General capabilities were not affected, according to Nous. The company argues that political bias in Chinese open models is only a thin layer.

Meanwhile, a researcher at the Anthropic Institute, Andy Hall, has looked at AI models used as auditors of other agents. They can spot most problematic behavior in agent logs. But the harder they try to catch every case, the more harmless actions they flag. Hall concludes they are better suited to estimating how often things go wrong than to real-time safety checks.

In brief. Open models are catching up on offensive cyber skills, according to the evaluation firm Vals AI. In a simulated corporate network, Deepseek V4.1 Flash completed 72.8 percent of objectives, against 80.8 percent for a restricted OpenAI model, at a fraction of the cost.

And money is getting tighter at the top. Anthropic still plans to go public before Thanksgiving, at a valuation of 1.25 to 1.75 trillion dollars, below what some investors hoped, after a tech selloff this week. OpenAI has received a first tranche of about 11 billion dollars, in a round now valuing it at 1.25 trillion, down from its 1.4 trillion target.

That's it for today. See you tomorrow.

Subscribe

RSS feed Apple Podcasts, Spotify