Introduction

Voice Arena, the independent speech-eval group behind the STT, TTS and diarization leaderboards, has launched Dictation Bench. It's a benchmark of the actual dictation apps people install, not the speech-recognition APIs underneath them. The team played the same 761 recordings from 87 speakers into 10 apps on a Mac (US English). Five LLM judges then edited each draft until it was ready to send. The headline metric is Zero Edit Rate (ZER): how often a draft could be sent with no changes. Wispr Flow leads at 79.4%. Gemini Dictation (75.8%) and Willow Voice (75.7%) are close behind, and Apple Dictation is last at 34.6%. The leaderboard and whitepaper are live now at voicearena.com/dictation-bench.

Sources: the Dictation Bench leaderboard with methodology and whitepaper links, and Voice Arena's Shobhit Banga's

(November 11, 2026).

The Leaderboard

Zero Edit Rate counts a draft as zero-edit when 3 of 5 judges change nothing. AMEND is the share of characters Claude Opus 5.5 changed when a fix was needed (lower is better).

  • 1. Wispr Flow: ZER 79.4%, AMEND 0.40%
  • 2. Gemini Dictation (Google): ZER 75.8%, AMEND 1.22%
  • 3. Willow Voice: ZER 75.7%, AMEND 1.88%
  • 4. Granola: ZER 71.7%, AMEND 1.04%
  • 5. Meta AI: ZER 69.4%, AMEND 2.54%
  • 6. ChatGPT Dictation (OpenAI): ZER 64.3%, AMEND 1.15%
  • 7. Claude Dictation (Anthropic): ZER 64.0%, AMEND 0.93%
  • 8. Superwhisper: ZER 59.8%, AMEND 1.44%
  • 9. Perplexity Dictation: ZER 52.7%, AMEND 1.85%
  • 10. Apple Dictation: ZER 34.6%, AMEND 2.60%

The two metrics disagree in useful ways. Claude Dictation sits mid-table on ZER but posts the second-smallest AMEND, so its drafts need fixing more often than the leaders' but need only light fixes. Meta AI and Willow show the reverse pattern. Latency is listed as "coming soon".

How It's Scored

  • Audio: 761 clips, 87 speakers, US English. The clips cover seven domains: AI prompts, code, docs/notes, email, personal messaging, work chat and long form. Each is tagged by length (micro to long) and difficulty, and the leaderboard can be filtered by all three. The audio stays private apart from two samples.
  • Apps: each run once, as shipped, on macOS. Free and paid plans are mixed.
  • Judges: Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon, Muse Wonder and Kimi K3.1 Pro each fix every draft. Punctuation and rewording are free, and filler words ("like", "you know", "uh") are penalized only when they're used as fillers. Voice Arena says no judge ranked its own lab's app above the panel's verdict.
  • Open parts: Voice Arena says the method, judge prompts, reference policies, app configurations, scores and code are public, and that gaps inside statistical ties are flagged.

The Failure Cases Worth Stealing

The thread's per-clip examples work as a ready-made test script for your own setup:

  • "Why is Redis memory sitting at ninety percent?" Only ChatGPT typed Redis. The others, Wispr Flow included, wrote Reddy's, Ready's or Randy's.
  • "Pydantic": ChatGPT, Gemini, Meta AI and Willow got it. Wispr wrote PineDantic, Perplexity Pindemic, and Claude and Apple pandemic. Code was the hardest domain for 4 of the 10 apps.
  • "Pin TanStack Query to version five point eight, sorry, five point six." 7 of 10 got it. Claude and Apple typed 10 stack query.
  • "Locker combo is, it's like twelve, thirty, eight." Only Gemini produced 12-30-8.
  • Over-helpful apps: Willow heard "Write me a Python script" and wrote the script, 1,814 characters nobody said. Meta AI drafted a review reply instead of transcribing the request for one.

Quick self-test: read those four lines into your own dictation tool. Then check whether self-corrections ("sorry, five point six") get resolved, and whether the app ever answers you instead of transcribing.

Why It Matters

Dictation is having a moment. This week alone brought ElevenAI's desktop dictation app on Scribe v3, Firefox's on-device Web Speech, and a wave of open local dictation tools. Until now the comparisons were vendor numbers or raw-model WER tables, which ignore the cleanup, formatting and "AI rewrite" layers that decide what actually lands in your message. Dictation Bench scores the whole harness, which is what people actually experience, and it already shows that a strong model doesn't guarantee a strong app. ChatGPT's dictation nails developer jargon but ranks sixth overall.

Caveats:

  • Scope is narrow: macOS, US English, one pass per app, free and paid tiers mixed.
  • Judges are LLMs with no human gold set yet, and the audio is private, so outsiders can't fully re-run it.
  • Voice Arena is a commercial eval and dataset company that also sells training data, so treat it as a strong independent signal, not a neutral standards body.
  • More apps are coming: Aqua Voice, Kivi, Typeless, Monologue and Spokenly are being evaluated for the next version.

Conclusion

Dictation Bench is the first public scoreboard for the dictation apps themselves, and its first answer is clear: Wispr Flow leads, Gemini and Willow are right behind, and the default Apple dictation trails badly. Use the domain and difficulty filters on the leaderboard to match your own use (code-heavy users should look closely), and run the Redis/Pydantic/TanStack lines through your app before you trust it with prompts.

—Anabel ♡