MaxText
1 article tagged with MaxText
November 19, 2026
researchGoogle
Google adds Streaming DiLoCo to MaxText, syncing 1/65th of a model per step for cross-datacenter training
Google DeepMind and Google Research engineers describe how MaxText implements Streaming DiLoCo, a low-communication optimizer that splits a model into fragments and syncs one at a time. In a Qwen3-32B worked example, data parallel exchanges ~65.5 GB per step, while Streaming DiLoCo exchanges ~1.0 GB per step.