Introduction

iFlytek has opened an API for SparkAudio 1.0, its speech foundation model, on the iFlytek Open Platform (讯飞开放平台). The company's announcement, posted by its Open Platform account on November 12 at 18:38 Malaysia time, says developers can call it today. It's an audio-understanding model: you send it speech (and a text instruction), and it sends back text.

That one model handles transcription, translation, dialect recognition, speaker separation, emotion analysis, environmental-sound recognition and open-ended questions about a recording. It covers 99 languages and 202 Chinese dialects. It's closed and cloud-only: there are no weights, and you call it over a WebSocket. iFlytek offers a free trial quota on the product page. We'd had it on our watch list since the October preview. This is the first time you can actually build on it.

What Shipped

  • Model: SparkAudio 1.0 (the API calls it spark-audio), from iFlytek. It replaces SparkAudio 1.0-Preview.
  • Input and output: audio plus an optional text prompt in, text out. The API docs say this version can't output audio, so it's not a TTS or voice-chat model.
  • Tasks: transcription, multilingual translation, dialect recognition, environmental-sound recognition, speaker separation, emotion analysis and complex audio Q&A. The announcement says it can also pick up a speaker's gender and age range.
  • Coverage: 99 languages, including Mandarin, English, Cantonese, Japanese, Malay, Indonesian, Thai, Vietnamese and Arabic, plus 202 Chinese dialects, from Cantonese, Hokkien, Teochew and Hakka down to city-level dialects like Hefei and Lanzhou. iFlytek publishes both lists: the languages and the dialects (xlsx).
  • One prompt, many jobs: you tell it what to do in plain text in the first frame. For example, "translate what this audio says" or "who sounds frustrated here?" There's an optional instructions field for a system prompt.
  • Companion model: iFlytek also announced Spark-ASR-2.0, a version of SparkAudio 1.0 fine-tuned for plain transcription. It's tuned for Mandarin, loud backgrounds and quiet speakers.
  • Trained at home: iFlytek says SparkAudio 1.0 was trained entirely on Chinese-made compute.

The Numbers

All figures are iFlytek's own, from the announcement. It doesn't say which test sets it used.

  • English speech recognition: 2.25% error rate.
  • 14 major languages: 2.81% average error rate.
  • Chinese dialects: 14.29% error rate.
  • Quiet Mandarin speech: 17.17% error rate. iFlytek says that's well below the models it compared against.
  • Versus the Preview: 20% better overall, 15% faster responses and 60% lower cost. iFlytek credits a "fast draft, then fix" decoder: a parallel decoder writes the transcript, then a language model corrects only the parts that need it.

Where to Run It Today

  • Sign up and get keys: open the product page to claim the free quota or buy a plan. Then get your AppID, APIKey and APISecret from the Spark-Audio console.
  • Endpoint: wss://tokens.xf-yun.com/omni/v1/realtime, with the header Authorization: Bearer YOUR_APIKEY.
  • Audio format: 16 kHz, 16-bit, mono PCM or WAV. iFlytek suggests sending 32,000-byte chunks every 40 ms.
  • The flow: send input_audio_buffer.append events (your text prompt goes in the first one only), then input_audio_buffer.commit to start inference. The answer streams back as response.text.delta events, then response.text.done with the full text. response.done also reports token usage.
  • Official demos: iFlytek has Python and Java samples.
  • Minimal Python sketch, built from the event protocol in the docs (pip install websocket-client):

import base64, json, uuid, websocket

ws = websocket.create_connection(
    "wss://tokens.xf-yun.com/omni/v1/realtime",
    header=["Authorization: Bearer YOUR_APIKEY"])
print(json.loads(ws.recv())["type"])  # session.created

pcm = open("clip_16k_mono.pcm", "rb").read()
CHUNK = 32000
for i in range(0, len(pcm), CHUNK):
    ev = {"event_id": str(uuid.uuid4()), "type": "input_audio_buffer.append",
          "audio": base64.b64encode(pcm[i:i+CHUNK]).decode()}
    if i == 0:
        ev["text"] = "Transcribe this, label each speaker, and describe their mood."
    ws.send(json.dumps(ev))
ws.send(json.dumps({"event_id": str(uuid.uuid4()), "type": "input_audio_buffer.commit"}))

while True:
    ev = json.loads(ws.recv())
    if ev["type"] == "response.text.done":
        print(ev["text"])
    elif ev["type"] in ("response.done", "error"):
        break
ws.close()

Caveats

  • Closed and text-out only. There are no weights, it's cloud-only, and it can't speak. Pair it with your own TTS if you need a voice reply.
  • No automatic turn detection. The API doesn't do VAD (voice activity detection), so your client decides when a clip ends and sends commit. That's fine for files, but it's extra work for live use.
  • Session limits: at most 300 seconds per append, 30 minutes of audio per session, 40 inference rounds, 120 minutes connected and 1 minute idle. Hit any of these and the session closes. For long recordings, chunk them and reconnect.
  • Pricing isn't in the docs. The announcement says calls cost "far below the market average", but it doesn't give a number. Check your plan on the product page before you scale up.
  • Vendor-only benchmarks. The error rates come from iFlytek's announcement, without named test sets. Test on your own audio, especially for dialects.
  • China platform. The docs and console are in Chinese. Inputs and outputs are content-moderated: a flagged prompt or answer returns an audit_blocked error and ends the session. iFlytek has been on the US Commerce Department's Entity List since 2019, so teams with US export-compliance rules should check before they integrate.

Conclusion

SparkAudio 1.0 is iFlytek's move from "hear clearly" to "understand": one API that turns a recording into a transcript, a translation, speaker labels, mood and scene notes, or a direct answer, across 99 languages and an unusually long list of dialects. If you work with Cantonese, Hokkien, Hakka, other Chinese dialects or multilingual speech for podcasts, interviews, game voice QA or call audio, it's worth trying on the free quota. Plan for client-side turn detection and the session limits, and keep a fallback if moderation or compliance is a concern.

—Anabel ♡