How to read the top model leaderboard: Who pays the most for AI inference and context?
Unlimited OCR2 could be your new scanning model. Genie 4 worries traditional video game developers. OpenAI produces proofs of major science theorems. Post-training method dispenses with human-written datasets.
In today’s edition of Data Points, you’ll learn about our top headlines, and more:
- Unlimited OCR2 could be your new scanning model
- Genie 4 worries traditional video game developers
- OpenAI produces proofs of major science theorems
- Post-training method dispenses with human-written datasets
But first:
Independent leaderboard purges saturated tests, reshuffles models
Artificial Analysis updated its Intelligence Index to version 5. The new index adds two benchmarks (FrontierCode 2.0 and Terminal-Bench-Science), upgrades three others, and removes three, so that 60% of the Index now rests on private test sets or held-out answers. The weights are split evenly across knowledge work, coding, science and general categories at 25% each. Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55), and no model leads all nine evaluations. The best open weights model, Kimi K3.1 Pro, scores 46 but its weights are not yet released; among released weights DeepSeek V4.1 Flash 1017 and GLM-5.3 lead at 32, below Claude Haiku 5.5 at 36. For developers, v5 shows that cost per task and effort setting can matter as much as the headline score: GPT-6.1 Sol (max) comes within 4 points of Opus 5.5 at about an eighth of the cost per task ($1.27 vs $10.04), and Opus 5.5 rises from 35 at low effort to 58 at max effort at 10 times the cost. (Artificial Analysis)
The Tokenizer Tax: Why some countries’ developers pay more
A study from the Coalition for Language Equity in AI Research found that AI models need far more tokens to process some languages than others, which the authors call a “tokenizer tax.” Among the 20 most common internet languages, the costliest used 1.5 to 5 times more tokens than the cheapest; across 200 languages, the gap in one model grew by 17 times. Script mattered as much as how rare a language is: Asturian, with few speakers and no Common Crawl data, benefited from its Latin script and related Romance languages, while Burmese, Tibetan, Tifinagh, Khmer, Lao, and Ol Chiki were consistently disadvantaged across nearly all models tested. More tokens also mean higher latency and less room in the context window for documents or conversation history. Because per-token pricing is the same for every language, developers building products in high-token languages pay more per request and get less usable context. The researchers and startup founders quoted say fixing it requires labs to design tokenizers for those scripts, not just add training data. (Rest of World)
Baidu’s open OCR model impresses on long documents
Baidu released Unlimited OCR 2, an open-source 3.34-billion-parameter vision-language model that parses documents of hundreds of pages in a single request. Its main innovation is SR-SWA (Sparse Reference Sliding Window Attention), which limits each decoding step to a fixed set of 512 reference tokens, so per-step latency and memory stay nearly constant as documents grow; image encoding and total runtime still rise with page count. Weights are available under the MIT license. BF16 weights total about 6.7 GB, and Baidu tested on one NVIDIA A800-80GB. Baidu’s own benchmark chart shows 84.6 on MPDocBench and 77.1 on MarathonDocBench, where the next model scores 33.7 and Baidu’s June predecessor scores 0.0. However, Baidu’s numbers are self-reported, and no public MarathonDocBench release was found. Teams considering it for book-length parsing should check those figures on their own documents first. (MarkTechPost)
Google Genie video and game simulation model now in testing
Google DeepMind announced Genie 4, a world model that generates interactive simulations in real time, now in research preview for trusted testers. Users navigate with keyboard controls and steer the scene with text prompts, such as changing the weather or adding a person. Genie 4 is the first Genie model to generate audio natively, including ambient sound and dialogue that responds to user input. In a separate setup, Gemini 3.8 Flash runs alongside Genie, breaking goals into steps and checking incoming frames to confirm each step succeeded. DeepMind has also tested Gemini 4 Argon with Genie for robotics-related scenarios. DeepMind lists limits that remain: visual details and object permanence drift over longer sessions, and complex physics, legible text, spatial accuracy, and audio accuracy are still active research areas. Access is not yet open to developers, but Google AI Ultra subscribers (through Project Genie) and early access partners in the Gemini API and Google AI Studio are next. (Google)
OpenAI research produces major scientific findings
OpenAI announced three results it says its models produced: a proof of the local smoothing conjecture in all dimensions, a counterexample that disproves the Exponential Time Hypothesis (ETH), and an existence proof for four-dimensional Yang–Mills theory and QCD. The math and computer science results have Lean formalizations and written proofs on OpenAI’s GitHub, but the physics group, led by David Gross, is withholding the AI-produced artifacts and writing the result up itself. OpenAI says it seeks no academic credit, imposes no nondisclosure agreements, and is funding a conference on the ETH result plus a $5 million Simons Foundation program for researchers examining AI-produced results. OpenAI calls these manuscripts raw research data, not established science; independent verification by the expert groups is still under way, and a Lean proof is only as meaningful as the statements it formalizes, which reviewers will need to check. (OpenAI)
Research method lets models generate own post-training datasets
A research paper introduces On-Policy Self-Training (OPST), a method for post-training a foundation model with no human-written questions or answers. Given only coarse, high-level topic descriptions, the model generates its own questions, answers, and privileged reasoning traces (reasoning written with access to the answer), then trains on them through on-policy self-distillation. A pseudo-verification filter keeps a problem only if the model can reproduce its answer in a fresh context and under a paraphrase of the question. The authors report consistent gains on standard language benchmarks, retained performance on domains the model was not self-trained on, and transfer to Visual Question Answering through a multimodal variant called Visual OPST. If these results hold up, fine-tuning a model on a new domain may not require a curated dataset. (arXiv)
Want to know more about what matters in AI right now?
Read the latest issue of The Batch for in-depth analysis of news and research.
Last week, Andrew talked about how engineering leaders could help their teams become AI-native by focusing on individual development, executing promising projects, investing in AI infrastructure, and redesigning processes around AI's new bottlenecks.
“Encourage your team to become skilled at AI Engineering, provide space and budget for experimentation, and set a culture that encourages this growth. Additionally, many engineering leaders are themselves hands-on builders or have returned to being more hands-on since coding agents have made it easier.”
Read Andrew’s letter here.
Other top AI news and research covered in depth:
- Meta released Wonder and opened up Spark, with three new models: Muse Wonder (available via API), Muse Wonder Full (subscription-only), and Muse Spark Open (weights free to download).
- OpenAI’s ChatGPT for Teens receives a report card: The company added new features while studies questioned the program's impact on safety and education.
- Nvidia courts a new partner as Reflection releases Beam, an open weight model from an independent U.S.-based lab.
- A single decoder transforms small model outputs into action with TurboVLA, which utilizes an old-school architecture to attend to language and vision.
A special event for our community
AI Dev brings together developers who build with AI every day. You'll hear from engineers at the companies shipping agents, models, and infrastructure, then meet them in person on the demo floor. Join us in New York City on November 30 and December 1.
Data Points is produced by human editors with AI assistance.
