Dear friends,

I previously wrote about our AI Engineering Skills Map, with the highest level skills being (i) Building and deploying AI applications, (ii) Software engineering fundamentals, (iii) Using coding agents, and (iv) Shaping the build. In this letter, I will flesh out the first of them.

Being skilled at building and deploying AI applications means knowing:

• LLM foundations
• Grounding models with data
• Building agentic systems
• Evaluation-driven development
• Operating in production
• Machine learning foundations

This map of skills was formed by analyzing a large number of job postings, structured expert interviews, and survey responses.

The key difference between AI applications and non-AI software is that the former’s output is less predictable. You don’t know in advance what an LLM will output, or what predictions a supervised learning algorithm will make. Because of this uncertainty, building AI systems is a much more iterative process than building traditional software — it is harder to plan the process in advance. Skilled AI engineers repeatedly build a piece of software, examine it, and decide what to try next, taking a sequence of steps that are highly influenced by the intermediate results. Being able to skillfully decide what to do next allows you to create reliable software systems based on unreliable AI components. This requires knowing:

LLM foundations. Understanding how large language models tokenize input and generate output allows you to understand when to count on them and when they may fail. It also allows you to understand when to use a multimodal model, how to make tradeoffs on what to include in the context window, and reason about cache hits, knowledge cutoff, reasoning effort level, sampling parameters, and when to use special features such as tool calling. Understanding these foundations helps you choose the right model or mix of models and apply specialized techniques when needed, such as fine-tuning or self-hosting models.

Grounding models with data. LLMs require good input context to produce useful outputs. RAG using vector search was an early attempt to give LLMs relevant context, but the set of techniques for grounding models with data has grown significantly. For example, you will have to decide what to include in a prompt vs. what to let an LLM retrieve on demand using tools, and which representation fits the data and search queries: a vector index, a knowledge graph, or a semantic layer over structured data (such as customer records). You’ll also turn documents (text, PDFs, HTML, images) into LLM-ready inputs and engineer pipelines to keep data clean and fresh. When you understand the menu of techniques available to get data, you are better able to give your LLM relevant context.

Building agentic systems. Agentic systems range from workflows that execute a predefined sequence of LLM calls to ones based on an agent harness that lets an LLM repeatedly decide its own next step. You’ll have to choose the architecture — what steps to chain, what to parallelize, when to use code and when to use an LLM — and engineer the workflow or harness, with fallbacks. When designing the agent loop, you will also decide what tools the model can call (including MCP, CLI and sandbox execution environments), what memory architecture to use, how to manage context over long sessions, and when a task needs multi-agent orchestration instead of a single-agent architecture. You’ll also want to turn promising prototypes into reliable, safe and secure agents for production; this requires understanding guardrails, adversarial inputs, and identifying and working around key risks (such as data exfiltration), and governance.

Agentic workflows are evolving rapidly, and you will also benefit from understanding any cutting-edge techniques relevant to your application area, such as voice agents, computer-use agents, or generative UI.

Evaluation-driven development. In my experience, the most important trait that distinguishes someone great at building AI systems is whether you can drive a disciplined evals/error analysis loop to drive development. This allows you to repeatedly focus your effort on directions that are more likely to be fruitful. I’ve found this to be a tricky skill to master, because the right approach varies significantly by project and even according to the stage of the project.

Building good evals is a deep technical skill. You might look at a system’s traces and outputs, carry out exploratory data analysis, and combine that with product and business insight to decide what to measure. You should also understand the menu of options for evals, such as when to use deterministic (code-based) evaluations, when to use an LLM-as-a-judge, and when to have a human in the loop, and how to evaluate your evals so as to keep evolving them. These evaluations then feed into an iterative process that drives further development, and makes progress systematic rather than random.

Operating in production. Operating AI software is different from traditional software because of its unpredictability, cost, and latency. First, you should know how to build observability mechanisms to understand the system’s performance on real usage. You’ll track performance, detect drift, and respond quickly to model failures and security incidents such as adversarial prompt injections. Putting in place regression testing and CI/CD requires more statistical evaluations than traditional software, and the testing effort should be calibrated relative to the risk of a mistake. Additionally, it’s important to know how to select the right mix of techniques — such as model choice optimization, distillation and fine-tuning, and agentic workflow simplifications — to optimize for cost and latency, especially if your application reaches many users.

Machine learning foundations. Modern LLMs are built using machine learning techniques including supervised learning and reinforcement learning. Every engineer I know that’s good at building with LLMs also understands machine learning and deep learning at some depth. Additionally, many applications still require knowing how to use machine learning – either a model someone else trained or one you train yourself. This requires knowing the popular machine learning and deep learning models and tradeoffs in accuracy, training speed, inference speed, and so on, and understanding how to engineer the data needed to train and evaluate these models. The machine learning concepts of bias/variance, error analysis, and engineering your data — all of which are core mental frameworks for navigating how to work with systems with uncertain output — also remain key to making a wide range of decisions in AI system development.

There is a lot to learn to become good at building and deploying AI systems. This is a field with significant technical depth. But every bit you learn will help you become better at AI Engineering and build more exciting applications. A strong complement to these skills is software engineering. I will write more about this in the next letter.

Keep building!

Andrew 

A MESSAGE FROM DEEPLEARNING.AI

We’re continuing to update the AI Engineering Skills map, and we’d love your input. Tell us more about what real AI engineers need to know to do their jobs. Complete the five-minute survey here

News

Grok’s Cursor Alliance Pays Off

Once a lab that produced mid-tier models, SpaceXAI has steadily improved. It just built one of the most capable models in the world while keeping prices relatively low. 

What’s new: SpaceXAI introduced Grok 4.6, a vision-language model developed with Cursor and aimed at long-running agentic work. It’s available to developers now via the API, in Grok Build and Cursor, and is due in the consumer Grok apps later.

• Input/output: Text and images in (up to 500,000 tokens), text out (no limit, 58.4 tokens per second)
• Knowledge cutoff: February 1, 2026
• Features: Adjustable reasoning levels (low, medium, high, xhigh — defaults to high reasoning), function calling, web search, X search, sandboxed code execution, a fast variant at double price
• Architecture: Roughly 1.5 trillion parameters
• Performance: Tied for third on Artificial Analysis’ Intelligence Index (61), second on GDPval-AA v2 and AA-Briefcase (1,577 Elo), top score on GPQA Diamond (94.9 percent)
• Availability/price: Via Cursor, Grok Build coding agent, Microsoft Office add-ins, GitHub Copilot, via API at $2.00/$0.50/$6.00 per million input/cached/output tokens with higher rates for requests beyond 200,000 tokens, fast mode $4.00/$1.00/$12.00 per million input/cached/output tokens
• Undisclosed: Architecture details, active parameter count, details of training data and methods 

How it works: Grok 4.6 is the latest model in SpaceXAI’s 1.5-trillion-parameter model family, building on Grok 4.5. SpaceXAI credits gains in performance to longer training on curated data, followed by fine-tuning on data generated by Grok 4.5 and reinforcement learning on agentic tasks. The training data included anonymized coding-agent data from Cursor, which included use of non-Grok models.

• The company pretrained the model on publicly available, internal, and licensed data. A further round of supplemental pretraining ran longer than Grok 4.5’s equivalent stage and used what the company calls an improved optimizer and training recipe. This data included synthetic data selected for reasoning, advanced technical concepts, and software engineering data to establish a stronger foundation for later fine-tuning.
• The company used Grok 4.5 to generate new training examples for fine-tuning, including transcripts of a model working through tasks. Grok 4.5 generated these transcripts for reasoning traces, agent harnesses, and tasks in STEM, software engineering, and knowledge work. Model-based filters removed flawed examples before Grok 4.6 trained on them.
• The company used human and synthetic reward signals for further reinforcement learning to fine-tune the model on knowledge work and coding tasks. It also used reinforcement learning to train the model in simulated environments for writing low-level GPU code (kernel optimization), building websites, and computer-aided design. 

Performance: Grok 4.6 improved its performance on both self-reported and independently measured benchmarks, rising to near the top of the leaderboards. On many benchmarks, Grok 4.6 rivals Claude Opus 5 and GPT-5.6 Sol and does so at a lower cost per task.

• On Artificial Analysis’ Intelligence Index, a composite of nine evaluations of economically useful tasks, Grok 4.6 set to high reasoning (61, $0.84 per task) ties for third place with GPT-5.6 Sol set to max reasoning ($1.23 per task).It jumps 5 points but more than doubles the price per task of its predecessor, Grok 4.5 set to high reasoning (56, $0.36 per task). Grok 4.6 ranks just ahead of Kimi K3 set to max reasoning (60, $0.84 per task) and just behind Claude Fable 5 set to max reasoning fallback (62, $3.14 per task).
• On GPQA Diamond, a test of graduate-level biology, physics, and chemistry questions, Grok 4.6 achieved 94.9 percent, the highest score among models that Artificial Analysis has tested.
• On Terminal-Bench 2.1, a test of command-line coding tasks, Grok 4.6 (88.4 percent) ranked third behind GPT-5.6 Sol set to xhigh reasoning (89.5 percent) and Claude Opus 5 set to max reasoning (89.1 percent).
• On AA-Briefcase, Artificial Analysis’ private benchmark of four multi-week knowledge-work projects that require an agent to use thousands of files across many turns to generate deliverables such as spreadsheets, presentations, and memos, Grok 4.6 set to high reasoning (1,577 Elo) trailed only Claude Opus 5 set to max reasoning (1,715 Elo), and it edged past Claude Fable 5 set to max reasoning with fallback (1,574 Elo). Grok 4.6 reached its results in about half the turns and one quarter of the input tokens as Claude Opus 5. Likewise, on GDPval-AA v2, which tests a model’s ability to generate a single deliverable such as a document or spreadsheet, Grok 4.6 set to high reasoning (1,746 Elo) trailed only Claude Opus 5 set to max reasoning (1,849 Elo).
• On τ³-Bench Banking, a test in which an agent resolves customer requests by searching policy documents and using tools, Grok 4.6 set to high reasoning (50.7 percent of tasks passed) trailed only Qwen3.8-Max (51.3 percent).

Behind the news: Grok 4.6 is the second model to come out of a partnership that led to an acquisition. In April, Cursor agreed to train its models on SpaceX’s Colossus supercomputer, a deal that gave SpaceX an option to buy the company. Cursor’s coding-agent data and SpaceXAI’s computation yielded results almost immediately: Grok 4.5, jointly trained with Cursor and introduced in July, lifted Grok 4.3 from 38 points on Artificial Analysis’ Intelligence Index to 56 points. SpaceX exercised its option in June, and the roughly $60 billion all-stock acquisition closed on August 14, days after Grok 4.6 launched. Three days later, Cursor introduced Origin, a code hosting service comparable to GitHub designed to handle the higher volume of code that agents generate. 

Why it matters: Model makers used to tout benchmark scores at launch. Increasingly, they also publicize cost and steps per task. Grok 4.6's clearest advantage over its near competitors is completing long-running work with fewer turns. At the same price per token and task, an agent that finishes in half the turns costs around half as much, which affects what applications are feasible to build with that model.

We’re thinking: The Cursor team brought data and technical expertise to SpaceXAI, and deserves credit for supporting Grok's rapid rise in model capability. Let’s hope Grok’s continued progress and aggressive pricing makes other top labs follow suit to keep per-token prices in check.

How Claude's Watermarks Work

Anthropic introduced invisible, machine-readable signals that text and images were generated by Claude.

What’s new: The marks will be deployed worldwide in all Claude models launched after August 2, 2026. The company said the watermarks are necessary in order to comply with the European Union’s AI Act.

How it works: Anthropic will generate a digital watermark – a hidden, coded signature that is used to identify AI-generated content – for generated text and a metadata credential for edited images. The policy applies to all forthcoming models and will be phased in for existing models. On August 14, the company added more details about its watermarking methods, their limitations, and implications for users.

• Claude’s technique is based on SynthID-Text, a method published by Google researchers in 2024. Typically, language models make low-stakes decisions between alternative words in a sentence. For marked text, a secret, randomized process – called the “seed generator” in the Google paper – subtly nudges the model toward certain word choices as it generates text. Those choices create a pattern that can later be detected by a scoring function, which measures how strongly the text matches the pattern. SynthID-Text gives a statistical score that indicates how strongly a piece of text matches the watermarking associated with its secret key. Anthropic says will release an API that assigns this probability score to submitted text.
• A watermarking signal gives a probability of Claude usage but it is not definitive evidence of generated text. Additionally, false negatives are possible in both watermarking and C2PA. Information created by humans but summarized, translated, condensed, or combined with synthetic information may contain a watermark signal.
• The company said watermarking will not (i) decrease output quality, (ii) be visible to the reader, (iii) require additional tokens or be more expensive, or (iv) have identifying information that could allow text to be traced back to a user.
• The watermarks are designed to be persistent. They survive copying and pasting and some editing. On the other hand, text that is heavily edited or paraphrased and images stripped of their metadata through format conversion may not include the watermark.
• Code and other deterministic text will generally have fewer watermarks. Unlike prose, there is frequently a best choice for code or more exact fields – for example, the correct next character for 2 + 2 = should inevitably be 4. But textual watermarks will still be used in cases where there isn’t a deterministic best choice and in code comments. This makes AI-generated code detectable.
• Claude models do not generate images from scratch but can edit and process images or generate them using code. These images will have a cryptographically signed credential in their metadata. The method, called C2PA, helps prove the provenance of an image or video. C2PA is different from a text watermark because, unlike in SynthID-Text, nothing in the content is adjusted. Any software that reads C2PA credentials can identify Anthropic’s, and the company will provide its own checking tool.

Behind the news: The announcement received widespread backlash, with many critics saying that watermarking is ineffective, harmful, or a privacy violation for AI users. Proponents argued that distinguishing between AI-generated and human-written text could actually help AI engineers.

• Anecdotally, dozens of Claude users on X claimed they canceled their Claude subscriptions. Still, Anthropic said it did not see a measurable increase in cancellations.
• Some users believe subtle adjustments in generated text will deteriorate output quality, despite Anthropic’s assurances to the contrary. Others worry about reputational damage associated with false positives and argue that the watermarking detection lacks nuance around how AI is used. For example, a person who uses Claude for editing assistance could be accused of entirely AI-generated messages; a lawyer could suffer backlash from judges or opposing counsel a flagged legal brief even if the content is accurate. Other professionals have similar worries.
• Some critics contend that watermarking isn’t meaningfully accurate since marked text can potentially be stripped away by passing text through a different AI model, or because one company’s API detector can’t detect text generated by other LLMs. Also, former Microsoft executive Steven Sinofsky said on X that users should have a “right to private thoughts free of a digital trail.”
• Other critics fear Anthropic will use watermarking to support claims of others copying or distilling their models, as part of Anthropic’s efforts to stymie competitors worldwide. 
• Proponents believe transparency around AI usage is a positive development. Scott Aaronson, a computer science professor whose work laid the groundwork for SynthID-Text, argued that even if detection methods aren’t perfect, watermarking can help deter scholarly cheating and plagiarism. And some AI researchers have argued that watermarks can be useful to avoid training unknowingly on synthetic content, which can lead to increased bias or model collapse in new generative models.

Why it matters: Anthropic isn’t the only company that will have to build a way for generated text to be detectable; it’s just the first mover. Article 50 of the EU AI Act demands machine-readable watermarks for generated content, including text, image, audio, and video. However, Anthropic chose to apply this globally, rather than only within the EU. Some large AI developers, including OpenAI, Google, Meta, and Microsoft, have also signed the EU’s Code of Practice on Transparency of AI-generated Content, a voluntary framework for demonstrating compliance with the AI Act’s requirements for marking and labelling AI-generated content. These and other companies may use different techniques, and it still remains to be seen if they will implement watermarking only in the EU, or globally.

We’re thinking: We’ve long taken the view that harmful use cases of AI should be regulated rather than the technology itself. Anthropic’s watermarking and detection methods are built into the technology, subtly altering generated text. Addressing plagiarism and identifying synthetic content for model builders may be laudable goals, but we anticipate that universal watermarking will prove to be too blunt an instrument, opening a Pandora’s box of issues around privacy, quality of output, and harmful false positives.

Qwen3.8-Max Lands With A Bang

Open models are getting larger and more capable. A few weeks ago, Moonshot AI announced Kimi K3, the largest and best-performing open weights model yet. Last week, Alibaba answered by releasing weights for a giant of its own.

What’s new: Alibaba first unveiled Qwen3.8-Max, a 2.4 trillion-parameter vision-language model trained to carry out long-running coding and knowledge work tasks, on August 2. The company released the weights of both Qwen3.8-Max and the smaller Qwen3.8-27B within a week. While smaller Qwen releases have been open weights, this is the first Max-tier model with downloadable weights. However, the open weights version of Qwen3.8-Max is limited to text input and output and doesn’t support the full million-token context window.

• Input/output: Text, image, and video in (up to 1 million tokens), text out (up to 131,000 tokens, up to 262,000 reasoning tokens, 77.6 tokens per second)
• Architecture: Mixture-of-experts transformer with hybrid attention, 2.4 trillion parameters total, 95 billion active per token
• Features: Reasoning (none, low, medium, or xhigh, with xhigh as default), reasoning text retained by default, function calling, structured output, prefix completion, context caching
• Performance: Fifth overall and second among open models on Artificial Analysis’ Intelligence Index (58), first overall on 𝜏³-Banking (51.3 percent), second on Arena.ai’s Vision Arena (1,301 Elo), fourth on Arena.ai’s WebDev Code Arena (1,667 Elo)
• Availability/price: API on Alibaba Cloud Model Studio at $2.00/$0.25/$6.00 per million input/cached/output tokens, subscription via QwenWork (currently free public beta in China only or via subscription in U.S.), weights for Hugging Face and ModelScope under a custom license
• Undisclosed: Knowledge cutoff, training data and methods

How it works: Alibaba has not yet published a technical report or model card for Qwen3.8-Max. The company says the model is built from Qwen3.5, its earlier vision-language model. Alibaba’s release notes for the model highlight reinforcement learning for agentic work and training that teaches the model to visually verify its own output using the model’s vision capabilities.

• Qwen3.8-Max’s mixture-of-experts transformer activates roughly 4 percent of its parameters to process each token. For Qwen3.5, hybrid attention combines standard attention layers with Gated Delta Network layers, a memory-efficient alternative to standard attention; Qwen3.8-Max closely follows that recipe.
• Like Qwen3.5, Qwen3.8-Max processes text and images jointly from the start of pretraining, rather than attaching a vision encoder to a trained language model. Alibaba says this design lets Qwen3.8-Max check rendered output as it plans, executes, and verifies its work.
• Alibaba also introduced QwenWork, an agentic harness designed to run the model.
• Alibaba fine-tuned Qwen3.8-Max via reinforcement learning in agentic environments, including QwenWork and other harnesses, that scaled along three axes: task length (single, multiple, or multiple days’ worth), workspace complexity (single file, multiple files, or hierarchical folders), and the harness in use (different tools, versions, and installed skills). One reward system graded every output by executing code, judging text and rendered images against rubrics, and inspecting results with agents. A data balancer kept each training batch’s mix of tasks, difficulty, and harnesses even to stabilize training.
• Qwen3.8-Max’s downloadable weights are text-only, and do not include vision input or the 1 million token context window. They are available under a custom license, similar to the MIT license, but requiring attribution for uses above 100 million monthly active users or $20 million in monthly revenue. Model-as-a-service or coding assistants built on Qwen3.8-Max with revenue over $50 million annually also must seek a separate license. Qwen3.8-27B’s weights include the model’s full capabilities and are covered by a standard Apache 2.0 license.

Performance: In independent evaluations at launch, Qwen3.8-Max trailed only a handful of top proprietary models and Moonshot AI’s Kimi K3 in overall intelligence. It likewise placed near the top in front-end coding, vision, and agentic work, leaping over Qwen3.7-Max on virtually all benchmarks. But these gains came at a price. The model generated more tokens than its rivals, which drove up its cost per benchmark task completed despite low per-token prices.

• On Artificial Analysis’ Intelligence Index, a composite of nine evaluations of economically useful tasks, Qwen3.8-Max set to reasoning (58, $1.13 per task) jumped 11 points over Qwen3.7-Max set to reasoning (47, $0.54 per task) to fifth overall rank and second among open-weights models. That put it above Claude Opus 4.8 set to max reasoning (57, $2.03 per task) and Muse Spark 1.2 set to xhigh reasoning (57, $0.40 per task) but behind Kimi K3 set to max reasoning (60, $0.86 per task).
• On Artificial Analysis’ 𝜏³-Banking tests of simulated customers' banking requests, Qwen3.8-Max achieved 51.3 percent, the best score of any model tested. On GDPval-AA v2, a head-to-head ranking of models’ work on tasks in economically central fields, Qwen3.8-Max achieved 1,739 Elo, behind only Claude Opus 5 set to max reasoning (1,846 Elo) and Claude Fable 5 set to max reasoning with fallback (1,743 Elo).
• On Arena.ai leaderboards, which rank models by human votes in blind head-to-head comparisons, Qwen3.8-Max debuted fourth (1,667 Elo) on WebDev Code Arena, behind Claude Opus 5 set to max reasoning (1,686 Elo) and Kimi K3 set to max reasoning (1,675 Elo). On Vision Arena, Qwen3.8-Max debuted second (1,301 Elo), behind only Claude Fable 5 (1,315 Elo).
• Qwen3.8-27B is currently third among open weight models with an Artificial Intelligence average of 52, comparable with GPT-5.6 Luna, and behind only . It is easily the most capable model capable that can currently run locally on a consumer machine.

Behind the news: Qwen 3.8-Max’s downloadable weights mark a strategic shift from Alibaba’s recent practice of keeping its most capable models to itself. Since the Max line began in January 2025, the company has offered those models, most recently Qwen3.7-Max, through its API and Qwen Chat app while releasing weights only for smaller models like Qwen3.6-27B. In Alibaba’s absence, other labs in China, including Moonshot AI and Z.ai, traded the open weights lead among themselves. Days after Kimi made its K3 model available, Alibaba previewed Qwen3.8-Max at the World AI Conference in Shanghai, before it made the model available via API on August 2. Z.ai followed with GLM-5.3 on August 14, a model that netted a 60 average on the Artificial Analysis Intelligence Index.

Why it matters: As always, open weights gives developers more options, from self-hosting to using Alibaba’s official tools or API. Admittedly, few teams can serve a 2.4 trillion-parameter model themselves, so the payoff will arrive via price-reducing competition among third-party hosting providers and distilled descendants like Qwen3.8-27B. Downloadable weights also let developers shop for better privacy terms, since any provider can serve them, including those that retain no customer data. 

We’re thinking: It’s disappointing that the open weights version of Qwen3.8-Max doesn’t include the full capabilities of the Alibaba API version, but Qwen3.8-27B on a laptop is a fun consolation prize — even if it does seem to overthink things sometimes.

Agents Come to Speech Recognition

Most speech-to-text systems transcribe speech in a single pass, which makes them unable to correct errors in their outputs. Researchers built a system that allows for interactive corrections.

What’s new: Zixuan Jiang, Yanqiao Zhu, Peng Wang, and colleagues at Shanghai Jiao Tong University, Zhejiang University, Fudan University, and Xiaoice, a Microsoft spinoff, devised a workflow that pairs an automatic speech recognition (ASR) engine with an LLM that attempts to detect and correct transcription errors at every conversational turn. They call it Agentic ASR. 

Key insight: Automatic speech recognition systems often manage mistakes by pairing a speech-to-text model with an LLM that rewrites its output. But correcting that system simply appends the user’s request for a correction to the existing transcription. Asking an LLM to rewrite a transcript from scratch risks introducing new errors, especially where the words spoken are idiosyncratic (for instance, the name Megan instead of Morgan). The LLM can more successfully revise the transcription if it divides the correction process into three steps: find the error, understand what the user said, and apply a fix. This approach enables the LLM to act more like an editor than a rewriter. 

How it works: Agentic ASR treats transcription as a multi-turn process of refinement, in which users can dictate and then confirm or correct the transcription in further turns.

• Given input speech, a speech-to-text engine (Qwen3-ASR-1.7B) generated its best guess at what the user said, and an LLM (Qwen3-32B) refined the guess using the transcribed output so far.
• The LLM classified the input into one of three intents: (i) confirmation (the user accepted the current state), (ii) new input (the user wanted to add to the transcription), or (iii) correction (the user wanted to alter the transcription).
• Confirmation or new input didn’t change the earlier transcription, but correction did. In that case, the LLM identified the span to edit, decided what the user wanted to change (for example “No, I meant Megan, not Morgan”), and applied the edit.

Results: The authors evaluated the workflow on multilingual speech-to-text benchmarks. Such benchmarks don’t include examples of corrections to output transcripts, so the authors used an LLM and text-to-speech engine to simulate a user dictating and then correcting the output text.

• The authors proposed a metric called S²ER that measures the proportion of transcription attempts that fail to preserve the speaker's intended meaning, which Qwen3-32B judged based on a ground-truth transcription. They compared the S²ER at 0 turns (equivalent to comparing against Qwen3-ASR-1.7B directly transcribing the speech) to the S²ER after the simulated user interacted with their system for 10 turns. Multi-turn interaction consistently reduced semantic errors across all benchmarks. Most gains appeared in the first few turns.
• On the GigaSpeech benchmark, the authors’ method improved S²ER from 21.5 percent at turn 0 to 3.5 percent after 10 turns, while word error rate improved modestly from 11.9 percent to 10.4 percent.
• On AISHELL-NER, which includes lots of names, dates, and the like, the authors method dramatically improved S²ER from 19.9 percent to 2.0 percent while cutting the Named-entity Error Rate from 2.4 percent to 1.2 percent.
• On ASRU2019, whose examples mix languages (for instance, English and Mandarin), the authors’ method decreased S²ER from 28.6 percent to 1.4 percent and improved Mixed Error Rate (which combines word error rate for English with character error rate for Mandarin) from 6.6 percent to 3.3 percent.

Why it matters: Many systems that accept voice input need to understand the user’s intent, not every word spoken. Agentic ASR does this very well, and when every word counts, it makes speech-to-text more robust in a natural way. This capability gains importance as UIs that depend on voice input become more widely used.

We’re thinking: The workflow of decomposing corrections into identifying where to edit, understanding what to change, and applying the change looks like a potential blueprint for interactive tasks such as editing documents, reviewing code, and iterating on a design.