Introduction

oruk, a small voice-analytics startup, has published open weights for orukqwen-asr-1.7b. It's an English speech-to-text model built by post-training Alibaba's Qwen3-ASR-1.7B with reinforcement learning (RL) instead of more supervised fine-tuning. The license is Apache-2.0. It uses the same architecture, tokenizer and prompt format as the base model, so anything that already runs Qwen3-ASR (Transformers, vLLM, the Qwen forced aligner) runs this one as a drop-in swap. The headline number: mean WER on nine public English Open ASR Leaderboard sets drops from 4.94% to 4.76%, with no speed penalty.

Sources: the model card on Hugging Face, oruk's research write-up Teaching a speech model what counts as wrong (dated November 10), and co-author Nathan Roll's

from November 11.

What Shipped

  • Model: oruk/orukqwen-asr-1.7b. It's called "1.7B" after its base, but it has about 2.04B parameters, stored as bf16 safetensors in the Transformers -hf layout.
  • Task: English speech-to-text for short clips (trained and tested on clips up to 30 s). oruk says recordings up to about 5 minutes transcribed fine in one call. Split anything longer at pauses.
  • License: Apache-2.0, inherited from Qwen3-ASR. No training audio is redistributed.
  • Where to run it: locally with Transformers (the card pins transformers==5.17.0) or vLLM 0.31, including vLLM's OpenAI-compatible /v1/audio/transcriptions endpoint. There's no hosted API or Space from oruk for this model.
  • Timestamps: word-level timing still comes from the unchanged Qwen/Qwen3-ForcedAligner-0.6B-hf.
  • Training cost: about 14.3 A100-hours on one 40 GB A100 at a time, plus at most 7.5 A100-hours for picking training clips. The RL pool was 20,000 English clips (54.6 hours of audio).

How the RL Recipe Works

Most ASR fine-tunes simply train on more transcripts. oruk instead lets the model grade its own guesses:

  • 64 guesses per clip: for each training recording, the model samples 64 candidate transcripts. Each gets a reward of 1 − WER against the human reference, and is scored against the other 63 using GSPO (Group Sequence Policy Optimization).
  • No credit for wrong words: a transcript can be "better than average" and still contain a misheard name. With plain RL, every token in that transcript gets rewarded, including the wrong one. oruk lines each guess up against the reference word by word and gives zero positive credit to substituted or inserted words. For dropped words it masks the next token, so skipping a hard word isn't rewarded either.
  • Soup, then blend back: three sibling RL runs are averaged into one set of weights (a "model soup"), then mixed 50/50 with the original Qwen weights (the WiSE-FT trick). You still download one model, not an ensemble.
  • The blend did the work: on its own, the RL soup scored 4.95% mean WER, slightly worse than the 4.94% base. Only the 50/50 blend got to 4.76%. oruk is open about this in the write-up.

Benchmarks (Vendor-Run)

These numbers come from oruk's runs of the official Open ASR Leaderboard harness, not from a leaderboard listing. The maintainers' private-set evaluation is still pending.

  • Nine-set mean WER: 4.94% → 4.76% (paired 95% CI −0.24 to −0.10 points, about 3.6% relative). A public H200 rerun on November 6 landed on the same 4.76%, with 718.84 RTFx aggregate speed.
  • Biggest wins: VoxPopuli 2.86% → 2.11%, URGENT2024 (noisy speech) 9.93% → 9.48%, AMI meetings 8.32% → 7.92%, GigaSpeech 7.22% → 7.11%.
  • Where it's worse: SPGISpeech 2.58% → 2.75%, a statistically significant regression. Earnings22 and Voice Arena Monsoon (Indian English) are flat.
  • Held-out English: oruk's own 18-set held-out mix drops from 10.09% to 9.09% pooled WER. AMI close-talk drops from 12.80% to 9.21%.
  • Speed: RTFx is within about 1% of the base on the same A100, as you'd expect from the same architecture.
  • Other languages: RL used English only. A small FLEURS/MLS check (de, fr, es, it, pt) showed no significant regression, but oruk still recommends the base model for non-English work.

Quickstart

From the model card (Transformers):

pip install "transformers==5.17.0" torch accelerate soundfile librosa

from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "oruk/orukqwen-asr-1.7b"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")

inputs = processor.apply_transcription_request(audio="meeting.wav", language="en").to(model.device, model.dtype)
out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[:, inputs["input_ids"].shape[1]:], return_format="transcription_only")[0])

Or serve it as an OpenAI-style transcription endpoint:

vllm serve oruk/orukqwen-asr-1.7b --max-model-len 2048 --port 8000
curl -s http://localhost:8000/v1/audio/transcriptions -F file=@meeting.wav -F model=oruk/orukqwen-asr-1.7b -F language=en

Raise --max-model-len (oruk used 8192) for clips of several minutes.

Why It Matters

The most useful lesson here is about transcript style, not the WER drop. RL taught the model to keep repetitions, false starts and "you know" verbatim. That helps on benchmarks whose references are verbatim (AMI, VoxPopuli, URGENT) and hurts on ones whose references are edited (SPGISpeech financial calls). oruk found that simply keeping or removing repetitions in the reference could flip which model ranked first, without the model hearing anything better. If you're picking an ASR model for meeting notes, captions or voice agents, check which style your app needs before trusting a leaderboard delta.

Caveats:

  • The gains are small in absolute terms (0.18 WER points on the nine-set mean), and every benchmark here is vendor-run.
  • Two known quirks: Chinese named entities or currency in English speech sometimes come out in Chinese characters (5 of 341 Earnings22 chunks vs 1 for the base). Casing and punctuation weren't trained.
  • The stronger follow-up candidate in the write-up (4.66% with beam search) is private and not part of this download. It's also about 3× slower.

Conclusion

orukqwen-asr-1.7b is a free, drop-in upgrade for English Qwen3-ASR users. It's slightly more accurate on meetings and noisy speech, it's more verbatim by design, and it runs at the same speed under Apache-2.0. Swap the model ID, test it on your own audio and your own transcript style, and keep the base model for non-English or heavily edited transcripts.

—Anabel ♡