Google makes Pathways, the distributed training system it built for Gemini, generally available on Google Cloud

Google announced the General Availability of Pathways on Google Cloud Platform on Nov. 9, 2026. Pathways is the distributed system Google says it built to train its own large models, including Gemini. A Google Developers Blog post by a team of Google engineers and product staff introduces the system and opens a series titled "Pathways on Google Cloud."

The core idea: one client, one graph

According to Google, Pathways addresses large-scale training as a coordination problem. One client describes the whole computation as a graph. Each block is a node, each tensor moving between blocks is an edge, and devices are where nodes are placed. A distributed runtime then places the graph, moves the tensors, and keeps the accelerators busy.

Where SPMD falls short

Google's post notes that most large-model training runs as Single Program, Multiple Data (SPMD) programs. Distributed Data Parallel (DDP) and Fully Sharded Data Parallel (FSDP) in PyTorch both follow this model. Every process runs the same script, owns its accelerators for the whole run, and coordinates through collectives such as AllReduce.

Google describes several limits:

• Interconnect gap: On a TPU v5p, each chip gets 1,200 GB/s of bidirectional chip-to-chip bandwidth inside its group. Bandwidth to the datacenter network is 50 Gb/s (about 6 GB/s), which Google calls roughly two orders of magnitude less. Because every step blocks on its collectives, spanning more groups takes a larger share of each step.
• Non-uniform jobs: Pipeline parallelism leaves devices idle in a "pipeline bubble" while earlier stages work. In RL post-training, the sampler, reward model and trainer are different programs that should overlap, but otherwise sit idle in turn.
• Failures: Google says hardware failures interrupted Gemini 2.5 training multiple times per hour. In SPMD, one lost machine stops every process and returns the run to its last checkpoint. Devices cannot be added or given back mid-run.
• Static capacity: Each process holds its accelerators from start to finish. A stage that needs fewer chips than another keeps the extras idle, and sharing a cluster means splitting it into per-job allocations.
• Debugging: Multi-host jobs need a launcher, and there is no single process to break into. When a job hangs, each worker is stuck at a collective, with no one place showing the whole job.

What single-controller execution changes

Google's proposal is that a single client declares the graph, including what each block does, what it needs, and where it runs. The runtime handles transfers, dependencies and ordering along the graph's edges. The post says readers will come away knowing where SPMD is suboptimal, what single-controller execution changes, and how Pathways coordinates a whole training job across a cluster.

Pricing, supported hardware beyond the TPU v5p example, and availability details were not given in the introductory post.

What this means

The post frames the problem as coordination, not raw compute. Google's argument is that once a job stops being one program in lockstep, as with pipelines and RL loops with separate sampler, reward and trainer programs, hand-wired per-process coordination becomes the bottleneck. Pathways is Google's answer, and General Availability means the system it uses for Gemini training is now offered on Google Cloud Platform. Teams running multi-stage or RL post-training workloads, or large runs where failures are frequent, are the ones the post's examples target. The rest of the "Pathways on Google Cloud" series is where implementation details should appear.