on-policy RL
1 article tagged with on-policy RL
November 16, 2026
researchGoogle
Google's Tunix hits bitwise-identical sampler and trainer logprobs for on-policy RL on 64 TPU v5p chips
Google engineers report a zero train-inference mismatch (Zero-TIM) implementation in Tunix, in which sampler and trainer assign bitwise-identical logprobs to the same token at the same weights. It was verified at multi-host scale across 64 TPU v5p chips (DP8×TP8). The recipe combines fixed-order Pallas kernels, explicit cross-shard reductions, a shared logprob path and custom differentiation rules.