Introduction

If your DAW session feels like five plugins fighting for the same pocket, StepFun (阶跃星辰) just handed producers a cleaner bus. On 15 September 2026, the lab launched the StepAudio 3 series—five audio models live on the StepFun open platform. Chinese trade desks clocked the drop mid-afternoon MYT (JRJ ~16:35, Sina ~16:51).

I’m leading with the music lane, not the chatbot lane. StepAudio 3 Music and StepAudio 3 Gen are the arrangement story. Realtime, TTS, and ASR are the voice stack that sits under it—same family, different faders.

Music First: Multi-Turn, Not One-Shot Karaoke

stepaudio-3-music-preview is built for the full arc—lyrics and style prompts into songs, dry vocals into accompaniment, covers, and iterative composition instead of “one prompt → pray.” Think less slot-machine chorus, more co-writer who stays in the room for another pass.

What producers should hear in the pocket:

  • Lyrics / style → full song with vocals, melody, arrangement, and mix intent in one go
  • Dry vocal / a cappella → intelligent scoring so your take gets a bed that follows the melody and mood
  • Covers and reference-driven runs when you bring timbre, humming, or a reference track
  • ABC-notation multi-turn iteration as the anti–one-shot move—shape the chart across turns instead of regenerating the whole universe every time

Platform docs list Music on HTTP async tasks (preview is free for a limited window). Structure tags like [Verse] / [Chorus] still help the model land sections the way a producer hears form—not as a flat blob of sound.

Gen: One Timeline for Speech, Vocals, SFX, Ambience, Music

stepaudio-3-gen-preview is the unified generator—the one that stops you bouncing five exports into one session. Natural language drives speech + voice design + vocals + SFX + ambience + music on a single timeline. Demo playground: stepaudiollm.github.io/step-audio-3-gen.

StepFun’s own blind-test framing puts Gen at Elo 1755.3 (TTS) and 1668.5 (Voice Design) on their leaderboard charts—useful as a vibe check, not a substitute for your ears. The creative win is compositional: describe a scene once, hear dialogue, room tone, hits, and underscore land in order instead of fighting stem sync later.

Pro Tip

Treat Gen like a sketch pad for sound design arrangement, and Music like the songwriting lane. Sketch the world in Gen (roles, director notes, SFX cues), then promote the musical idea into Music for lyric-led structure and multi-turn polish. Don’t force one model to be both the film mixer and the topline writer in the same take.

Voice Stack: TTS, Realtime, ASR Under the Same Roof

Once the music bed is swinging, the rest of StepAudio 3 locks the vocal and agent lane:

  1. StepAudio 3 TTS (stepaudio-3-tts) — human-like delivery with laughter, hesitation, and self-correction; streaming generate-and-play for low-latency reads.
  2. StepAudio 3 Realtime (stepaudio-3-realtime-preview) — native full-duplex; think-while-speaking; async tool calls. Artificial Analysis Conversational Dynamics 98.9% (#1) and Speech Reasoning #1 on their charts—the interruptible agent voice, not a turn-based robot.
  3. StepAudio 3 ASR (stepaudio-3-asr-max) — WER 1.7%, tied #1 on Artificial Analysis non-streaming ASR; Chinese/English, dialects, code-switching; holds up on whisper, fast speech, and background music.

Together that’s a producer-friendly stack: write and iterate songs (Music), build full soundscapes (Gen), cast expressive VO (TTS), run live voice agents (Realtime), and transcribe messy takes (ASR)—all listed side-by-side in the English audio models guide.

Papers landed slightly earlier on arXiv (Gen 2609.12945, Realtime 2609.14005); the product launch on the open platform is today.

How to Get Started

Practical session plan for generative producers:

  1. Open the model card — Start at platform.stepfun.ai audio docs and note the five StepAudio 3 IDs above.
  2. Hear Gen first — Spend ten minutes in the StepAudio 3 Gen demo. Write one scene with roles + director notes + a music bed cue; listen for timeline cohesion.
  3. Park a Music draft — Feed structured lyrics ([Verse] / [Chorus]) plus a concrete style (genre, tempo, vocal character). Then iterate—don’t one-shot and bounce.
  4. Score a dry vocal — Drop an a cappella and ask Music for accompaniment that follows the pocket, not a generic loop pack.
  5. Cast VO with TTS — Prompt for hesitation/laughter when the read needs human swing; stream-play while you edit the bed.
  6. Stress-test Realtime — Interrupt mid-sentence; check think-while-speaking and whether tool calls stay out of the musical path.
  7. ASR the messy take — Run a whisper-y or music-under vocal through stepaudio-3-asr-max before you chase lyrics by ear.
  8. Keep rights clean — Covers, clones, and reference audio still need clearance. Preview ≠ free commercial forever—watch the platform’s billing notes when the free window ends.

StepAudio 3 producer cheat sheetMusic: stepaudio-3-music-previewGen: stepaudio-3-gen-previewTTS: stepaudio-3-ttsRealtime: stepaudio-3-realtime-previewASR: stepaudio-3-asr-maxDemo: https://stepaudiollm.github.io/step-audio-3-gen/Docs: https://platform.stepfun.ai/docs/en/guides/models/audio

Conclusion

StepAudio 3 isn’t another isolated “AI singer” button—it’s a five-model rack where Music and Gen carry the arrangement story, and TTS / Realtime / ASR lock the voice lane underneath. Multi-turn music work and unified soundscape timelines are the heavy-impact moves for anyone living between lyrics, stems, and scene audio.

Open the platform, sketch one Gen world, iterate one Music draft, and decide which fader earns a permanent seat in your template. The low-end is there if you listen for it.

—Anabel ♡