MOSS-Image-heal: FP16-Safe Residual Finetune

Introduction
Hours-old on Hugging Face: KBlueLeaf/MOSS-Image-heal (created 2026-11-04 ~05:39 UTC) is a full transformer finetune of OpenMOSS-Team/MOSS-Image-Base aimed at one pain point — the base model's residual stream peaks near the FP16 cliff. Heal pulls the largest stream value from 58,411 down to 3,083, cuts stream RMS about 7.5×, and still tracks the base closely (flow loss 0.4289 vs 0.4280, velocity cosine 0.9983 on the card's probe set). Apache-2.0 weights, Diffusers swap, and ready ComfyUI single-file checkpoints in the same card.
What "heal" actually changes
Only the transformer is trained. Text encoder, tokenizer, VAE, scheduler, and pipeline stay the official MOSS-Image-Base pieces. The recipe is stream healing in the KohakuMoss sense: flow matching plus distillation toward a frozen base, plus losses that punish concentrated stream writes and high stream RMS, with a decoupled weight-outlier decay so no element sits above 64× its tensor RMS (base had 349 elements above that bar; heal has 0).
Training snapshot from the model card:
- 50,000 steps × 64 samples (~3.2M samples), ~200 GPU-hours on B300
- ~1M quality-ranked Danbooru posts with Graph-Based Captioning (GBC) graphs
- Prompt formats mix tags, natural language, and
gbc.softlayout graphs - Text refiner stays frozen; everything else in the transformer moves
Precision check on the card: the FP16 Comfy file peaks around 3,278 in the stream on the probe rows — roughly 20× headroom under the FP16 max — while base peaks near 61,696 on the same rows. That is the practical win for --fp16-unet graphs and INT8 ConvRot packs that used to flirt with overflow.
Where to run it today
ComfyUI (easiest maker path). Download one of the files under comfyui/ into models/diffusion_models/:
moss_image_heal_bf16.safetensorsmoss_image_heal_fp16.safetensors(pair with ComfyUI--fp16-unet)moss_image_heal_int8_convrot.safetensors(Comfy-Org INT8 ConvRot layout)
Keep the usual Comfy-Org Qwen3.5-9B text encoder and FLUX.2 small VAE you already use for MOSS-Image. Load with Load Diffusion Model in the official MOSS-Image workflow — no new custom node pack required.
Diffusers. Load the official MOSS-Image-Base pipeline, then swap in heal's transformer:
import importlib
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"OpenMOSS-Team/MOSS-Image-Base",
custom_pipeline="pipeline_moss_image",
trust_remote_code=True,
transformer=None,
torch_dtype=torch.bfloat16,
)
pipe.scheduler = type(pipe.scheduler).from_config(pipe.scheduler.config)
MOSSImageTransformer = importlib.import_module(type(pipe).__module__).MOSSImageTransformer
pipe.transformer = MOSSImageTransformer.from_pretrained(
"KBlueLeaf/MOSS-Image-heal",
subfolder="transformer",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")This is open weights (Apache-2.0, same as official MOSS-Image). Local Comfy / Diffusers today — not a closed API partner node.
Why it matters
MOSS-Image already has native Comfy support and a growing LoRA scene. Heal is the reliability patch for people who want FP16 / INT8 paths without the base stream slamming the float16 ceiling, while keeping samples almost interchangeable with base (the card shows side-by-side caption and GBC grids). If your MOSS graphs crash, wash out, or refuse to stay in FP16 under load, this is the checkpoint to A/B first.
Caveats
- Brand-new card (★1 at scout time) — treat as day-0 community finetune, not a vendor release
- Still needs the official MOSS text encoder + FLUX.2 VAE stack; heal does not replace those files
- License is Apache-2.0 on the heal weights; confirm any other stack pieces you mix in
- Not a Gen-hosted model on ArtRealmAI today — run local / your own host
Original Source
Conclusion
Drop moss_image_heal_bf16 (or the FP16 / INT8 ConvRot twin) into diffusion_models/, keep your MOSS encoder + small FLUX.2 VAE, and A/B against MOSS-Image-Base on the same seed. If the stream used to flirt with FP16 overflow, heal is the quiet fix.
—Aurelia ♡
