Introduction

If you wanted FLUX.2 [klein] 4B outside Python Diffusers, today is the day the graphs showed up. On Nov 3 (02:21 UTC), smile-forge published smile-forge/flux2-klein-4b-onnx on Hugging Face: the Black Forest Labs checkpoint (revision e7b7dc27) traced into four ONNX graphs aimed at ONNX Runtime 1.30+ with the CUDA execution provider.

This is open Apache-2.0 weights for local image-gen — text-to-image plus instruction edit, multi-image combine, and masked inpaint — not a hosted API. The pack was built for the Nexaflow Windows host (C# sampling), but any ONNX Runtime CUDA caller can load the same folders.

What shipped

Four graphs, with the sizes the card quotes (and the files that are live on the Hub right now):

  • encoder/ — Qwen3-4B prompt encoder cut after layer 27, 8-bit MatMulNBits weights, fp32 arithmetic — about 3.8 GB.
  • denoiser/ — rectified-flow transformer, 8-bit weights on fp16 branches / fp32 residual streams — about 4.6 GB.
  • picture-encoder/ — VAE encoder in fp16 — about 69 MB.
  • decoder/ — VAE decoder in fp16 — about 99 MB.

Concrete fidelity checks vs Diffusers’ Flux2KleinPipeline / Flux2KleinInpaintPipeline (bf16, same noise) are on the card: whole-picture instruction edits land around 30–37 dB PSNR, masked fills about 31–43 dB, border expansion about 26 dB, and prompt-only draws about 18 dB (same content, details moved). The decoder alone is ~63 dB from the fp32 VAE.

Build notes that matter for deployers:

  • Residual streams stay fp32 because single-stream blocks can hit ~1.03×10⁵ — past fp16 range.
  • Attention uses ORT’s fused com.microsoft.MultiHeadAttention.
  • Sampling matches the distilled Klein recipe: four Euler steps, no guidance.
  • Pictures stay within a megapixel; sides are multiples of 16.

Peak VRAM the README reports inside Nexaflow’s CUDA arena for a megapixel image: about 4.9 / 2.0 / 4.8 GiB for encoder / picture-encoder / decoder, and 7.3–13.4 GiB for the denoiser depending on how many reference pictures you feed. On an RTX PRO 6000 they clock roughly ~1 s per picture.

Where it fits

This drop is for people who want Klein’s edit stack in a non-Python or ORT-native host — desktop shells, game tools, or custom CUDA services — without re-implementing Diffusers. It is a conversion of the official weights, not a new base model, and it is not a ComfyUI node pack.

If you are already happy in Comfy or Diffusers, keep using the native Klein pipelines. If you need ONNX Runtime on CUDA with measured edit fidelity, this is the first clean public packaging we have seen for that path.

Run it today

  1. Open the card: https://huggingface.co/smile-forge/flux2-klein-4b-onnx
  2. Pull the four folders (encoder, denoiser, picture-encoder, decoder) — expect on the order of ~8.5 GB of graph + weight data.
  3. Load them with ONNX Runtime ≥ 1.30 and the CUDA EP. The README documents tensor names, dtypes, and the four-step Euler schedule.
  4. Optional: follow the same sampling path Nexaflow uses, or point your own host at the graphs. Build/check scripts (build_flux2_klein.py, check_flux2_klein.py, Build-Flux2Klein.ps1) are in the repo if you want to re-export.

Original Source

Conclusion

Klein already moved fast in Diffusers and Comfy — this ONNX Runtime CUDA cut is the “ship it into a C# / ORT host without rewriting the model” chapter. Four graphs, Apache-2.0, real PSNR numbers, and a runnable megapixel path today. If your stack speaks ONNX better than Python, this is the Klein edit kit to try next. ✨🖼️

Try this on Gen → https://artrealmai.com/gen?utm_source=magazine&utm_campaign=gen&utm_content=flux2-klein-4b-onnx-runtime-cuda-edit

—Aurelia ♡