Introduction

You can now train TRELLIS.2 on your own asset style without writing the training stack yourself. On November 12, Hugging Face published a fine-tuning guide for TRELLIS.2, Microsoft's open image-to-3D model. It comes with an MIT-licensed scripts repo and two ready-made containers, one for preprocessing and one for training. You point them at a Hub dataset of GLB or OBJ meshes and get LoRA adapters back on the Hub. In the authors' test, adapters trained on just 188 shoes moved held-out mean F-score from 0.276 to 0.803.

What shipped

  • What it is: a LoRA fine-tuning recipe for TRELLIS.2, written by Suvaditya Mukherjee, merve and Pedro Cuenca on the Hugging Face blog
  • Modality: image to textured 3D mesh (GLB)
  • Open vs API: open. The scripts are MIT, TRELLIS.2 itself is MIT, and training uses Hugging Face's PEFT library
  • Where to run it today: Hugging Face Jobs (hf jobs run) or your own GPU through docker run --gpus all, using the cr.hf.co/suvadityamuk/trellis2-finetuning-prep and trellis2-finetuning-train images
  • Concrete fact: about 9.3M adapter parameters across two DiTs. On nine held-out shoes, Chamfer distance fell from 0.174 to 0.033 and the F-score improved on all nine

Why it matters for game asset pipelines

General image-to-3D models get the overall look right and then miss the details your project cares about. In the blog's test, the base model made convincing shoes with wrong soles. Game teams see the same thing with their own categories, like weapon silhouettes, props built to a house style, or vehicles that need believable underbodies.

The recipe deliberately trains only the parts that control shape. TRELLIS.2 runs in stages. A sparse-structure flow DiT predicts a coarse 64³ occupancy grid, a shape flow DiT builds 512³ O-Voxel geometry, and a texture flow DiT paints the PBR materials. The guide adds rank-8 adapters to the sparse-structure DiT (attention and MLP) and rank-16 adapters to the shape DiT's attention layers. The texture stage stays frozen. The reason is practical: scanned datasets usually have diffuse maps with lighting baked in, and training the material stage on those would teach it to paint shadows into albedo. If your library has clean PBR maps, you could try that stage later. The guide doesn't do it.

Try it

You need a HF_TOKEN with access to the gated DINOv3 model, which TRELLIS.2 uses as its image encoder. Upload your meshes to a Hub dataset first, then preprocess them. The container renders 16 Blender views per object and converts each mesh into TRELLIS.2 latents:

hf jobs run -d --flavor a100-large --timeout 6h --secrets HF_TOKEN \
  cr.hf.co/suvadityamuk/trellis2-finetuning-prep:latest \
  trellis2-prep --meshes your-username/my-meshes \
  --push-to your-username/my-meshes-trellis2 \
  --private --holdout 0.05 --views 16 --workers 8

Then train one adapter per stage. ss targets the sparse-structure DiT and shape targets the shape DiT:

hf jobs run -d --flavor a100-large --timeout 12h --secrets HF_TOKEN \
  cr.hf.co/suvadityamuk/trellis2-finetuning-train:latest \
  trellis2-train --config shape \
  --latents your-username/my-meshes-trellis2 \
  --push-to your-username/my-trellis2-lora \
  --trackio-space your-username/trellis2-trackio

To test the pipeline before you prepare any data, swap in their preprocessed footwear latents (suvadityamuk/gso-footwear-trellis2-latents). Interrupted jobs resume, because preprocessing skips objects that are already finished and training continues from the newest checkpoint on the Hub.

The catch

  • It isn't cheap. Each run used one A100 80 GB. Preprocessing the 188 objects took about an hour, the sparse-structure adapter took 7.2 hours and the shape adapter took 3.6 hours. The guide doesn't give a minimum VRAM for local Docker runs.
  • The proof is narrow. The numbers come from one category (footwear from Google Scanned Objects) and nine held-out objects. Whether it works as well on stylized, low-poly or hard-surface game assets is your experiment to run.
  • It tunes geometry only. Materials still come from the base model.
  • Watch your data license. The adapters learn from whatever meshes you feed in, so use assets you own or have cleared for training.

For studios with a few hundred in-house meshes in a consistent style, this is the most direct path so far to a TRELLIS.2 that generates in that style. You also get an evaluation loop with Chamfer distance, F-score and normal consistency, plus Trackio logging for every run.

—Titus