China's open AI models fall 26 points behind as new index hides the answers
What's the deal? Artificial AnalysisDealroom has a profile for this one. Try Dealroom → released version 5 of its Intelligence Index on October 27, 2026, and the overhaul sharply widened the gap between closed and open-weight AI models. The leading open model now sits 26 points behind the overall leader. It is the benchmarking firm's biggest update since v4 launched in January.
What changed? For the first time, 60% of the index draws on private test sets or held-out answers, a move meant to stop models from training on public test questions. The index weights four equal categories — knowledge work, coding, science, and general knowledge — and adds new private benchmarks, including CognitionDealroom has a profile for this one. Try Dealroom →'s FrontierCode 2.0 and Terminal-Bench-Science.
The results: Claude Opus 5.5 leads at 58, ahead of OpenAI's GPT-6 Astra at 57 and GPT-6.1 Sol at 55. Anthropic and OpenAI hold the top five spots.
Open models take a hit: Kimi K3.1 Pro from Moonshot AI remains the best open model at 46, down from 55, though its weights are not yet released. Among models with released weights, DeepSeek V4.1 Flash and GLM-5.3 lead at just 32 — 26 points behind the leader, and below Anthropic's small Claude Haiku 5.5 at 36.
Why now? The redesign targets benchmark contamination directly. On the revised Humanity's Last Exam, Kimi scores 37% against Opus's 68%; on scientific agent tasks, Kimi reaches 34% while most open models stay below 10%.
What could go wrong? The result reverses the near-parity picture from a September Mozilla analysis, which found Chinese open models roughly four months behind frontier US offerings. But the two are not directly comparable, since both the index and its evaluations changed with v5.
The cost angle: The harder tests demand far more compute. Models now generate a median of about 59,000 output tokens per task, two-thirds more than before, pushing a single Claude Opus 5.5 run to roughly $10 per task. Across the top 10 models, cost per task varies by more than 13x.
The signal: The widened gap reflects this particular, redesigned benchmark rather than proof that Chinese open models have broadly fallen behind. As evaluators shift toward private data, the story becomes less about which model tops a leaderboard and more about how much of past parity rested on public test answers.
Read more: artificialanalysis.ai, trendingtopics.eu
