NVIDIA: FLUX 3 Image Klein Base runs up to 3.46x faster on one B200 with TensorRT LLM VisualGen

NVIDIA reports that FLUX 3 Image Klein Base generates images up to 3.46x faster (text-to-image) and 4.45x faster (image-to-image) at 2048x2048 on a single B200 GPU when TensorRT LLM VisualGen optimizations are combined, compared with uncompiled BF16 inference. The figures come from an NVIDIA developer blog post published November 6, 2026, and are NVIDIA's own measurements.

FLUX 3 Image Klein is an image generation and editing model from Black Forest Labs (BFL), trained on NVIDIA GB200 NVL72 systems. It ships as a Base checkpoint and a distilled checkpoint. The architecture combines Qwen3-VL-4B-Instruct for text conditioning, a diffusion transformer (DiT) that refines image latents, and an autoencoder that decodes latents and encodes reference inputs for image-to-image (I2I).

Where the time goes

NVIDIA's baseline uses NVFP4, VANILLA attention and torch.compile at 2048x2048 on one B200, averaged over 10 requests after two warmups. Base runs 50 denoising steps with a CFG scale of 3.5. The distilled model runs 16 steps without CFG.

Task	Model	Denoising	Total	Denoising share
T2I	Base	14.24 s	14.70 s	96.8%
T2I	Distilled	2.28 s	2.71 s	84.0%
I2I	Base	39.40 s	40.29 s	97.8%
I2I	Distilled	6.65 s	7.48 s	89.0%

Because denoising dominates, NVIDIA focuses its optimizations on the DiT.

Reported speedups

• NVFP4 vs BF16: up to 1.62x faster T2I and 1.58x faster I2I on Blackwell. VisualGen supports BFL's prequantized NVFP4 DiT checkpoints, enabled with variant: nvfp4 in pipeline_config.
• GEMM kernels: FlashInfer's PrimsTS FP8/NVFP4 kernels, enabled by default, delivered 1.02–1.10x faster T2I on B200 than the autotuned TensorRT LLM NVFP4 GEMM at launch. They fuse GEMM with SwiGLU, and QKV projections with query/key normalization and RoPE.
• Quantized attention: VC-Attention FP8 with a repair budget of 0.005 gave 1.18x speedups for both Base and distilled at 2048x2048, with LPIPS scores of 0.086 and 0.151. cuDNN FP8 is also available.
• Sparse attention: Sol-Attention with τ=1.0 and 12 initial dense steps gave 1.30x faster denoising plus decoding for Base. Using 16 dense steps favors fidelity at 1.24x.
• Step caching: Cache-DiT nearly halved Base latency at 1024x1024 and 2048x2048, reaching 1.94x at the higher resolution with an LPIPS of 0.033.
• Multi-GPU: CFG and Ulysses parallelism on eight B200 GPUs cut median T2I latency from 14.90 to 3.00 seconds for Base and from 2.73 to 1.08 seconds for distilled. Denoising speedups were 6.01x and 4.44x.

Unless noted, benchmarks use torch.compile and seed 0.

Deployment

The tutorial includes an end-to-end FLUX 3 example with YAML configurations. NVIDIA directs users to the TensorRT LLM VisualGen runtime and optimization utilities, and invites issues and results on the TensorRT LLM GitHub.

What this means

For Base, denoising accounts for 96.8% of text-to-image time and 97.8% of image-to-image time in NVIDIA's baseline, so DiT-level changes (low-precision GEMM, attention, caching, parallelism) are where the reported gains come from. The distilled checkpoint at 16 steps without CFG starts from a much lower latency than Base, and multi-GPU scaling brings both closer together in absolute terms.

Several techniques trade output change for speed, and NVIDIA reports LPIPS for attention and caching options, so teams can weigh fidelity against latency per workload. All results are single-vendor measurements on B200 hardware with fixed seeds and settings, so latency on other configurations will differ.