DiffusionGemma

Google DeepMind published a technical report on DiffusionGemma, a text diffusion model built by retrofitting Gemma-4-26B-A4B rather than training from scratch. The model generates 256-token blocks in parallel, reaches about 1,500 tokens per second on an Nvidia H100, and uses less than 10% of the original training budget.

NVIDIA released a quantized version of Google DeepMind's DiffusionGemma 26B A4B IT, a multimodal model with 25.2B total parameters (3.8B active) that processes text, image, and video inputs. The NVFP4-quantized model achieves generation speeds exceeding 1,100 tokens per second on NVIDIA H100 GPUs while supporting a 256K token context window.

Google DeepMind released DiffusionGemma, a 26B parameter mixture-of-experts model that generates text using discrete diffusion instead of autoregression. The model processes blocks of 256 tokens in parallel, achieving generation speeds exceeding 1100 tokens per second on H100 GPUs in low-batch settings.