Tunix agentic GRPO lifts Gemma 4 E2B from 9.4% to 95% win rate in Plane Strike
Google's developer blog details how Tunix's agentic GRPO pipeline trained Gemma 4 E2B to play Plane Strike with tool calls. The win rate on held-out boards rose from 9.4% to 95% after 100 steps on a single Cloud TPU v6e-8 host.
Google's Tunix agentic reinforcement learning pipeline raised the Plane Strike win rate of google/gemma-4-E2B-it from 9.4% to 95% on held-out boards after 100 steps of agentic GRPO, according to a Google developer blog post published November 19, 2026 by developer advocate Wei Wei. Training ran on a single Cloud TPU v6e-8 host.
The results
Out of the box, Gemma 4 E2B in thinking mode wins 9.4% of games on held-out boards, averaging 22.44 strikes per game. After 100 steps of agentic GRPO (Group Relative Policy Optimization) with Tunix, the win rate reaches 95% and the average game length falls to about 16 strikes.
The post describes the model as having decent spatial reasoning and tool-calling capabilities before training.
The task
Plane Strike is played on a 6x6 grid (A1 to F6). One 8-cell plane is hidden in one of 4 orientations. Each turn, the agent sees the result of its previous strike and the raw visible board (. untried, X hit, O miss), reasons through candidate placements, and calls strike(cell="...") to hit one cell. A game ends when all 8 plane cells are hit or after 23 strikes. The post revisits a game from an earlier TensorFlow/JAX reinforcement learning blog, replacing the custom MLP policy with Gemma 4 E2B.
Pipeline components
The pipeline has four modular parts:
- Task environment:
PlaneStrikeEnvsubclasses Tunix'sBaseTaskEnv. A seed fixes the hidden plane position so all 8 rollouts in a GRPO group play the same board. Each step returnsEnvStepResult(observation, reward, done, info), and step rewards are summed into a trajectory-level reward. - Tool and chat template parser:
StrikeToolextendsBaseTool, and a subclass ofGemma4ChatTemplateParserappends tool declarations to the system prompt and renders tool messages as in-turn responses. - Conversation agent:
PlaneStrikeAgentextendsConversationAgentBaseand maps environment observations to model messages. - Trajectory collection and learner:
TrajectoryCollectEngineandGRPOLearnerhandle rollouts and policy updates.
Gemma 4 tool-calling format
Gemma 4 declares tool schemas in the system prompt inside <|tool>declaration:...<tool|>. A model turn that calls a tool contains a thought channel followed by a <|tool_call>call:strike{...}<tool_call|> block, then <|tool_response>. The post identifies <|tool_response> as one of Gemma 4's native stop tokens: the model pauses there, receives the environment response, and continues its thought and next tool call within the same assistant turn.
The agent ignores everything inside the thought channel and parses only the final answer. Only a final answer containing a single valid strike call counts as a valid coordinate. Anything else returns INVALID_ACTION.
Rollout configuration
Training uses Tunix's asynchronous vLLM rollout engine with the JAX TPU backend, in server mode with prefix caching enabled and bfloat16 dtype. The configured KV cache size is the maximum prompt length plus the maximum response length plus 256. EOS tokens include the tokenizer EOS, <turn|> and the tool-response start token. The visible config also sets final_logit_softcapping to 30.0 through Hugging Face overrides.
For standard multi-tool agents, tunix.rl.agentic also provides ToolEnvironment, ToolAgent and ToolManager abstractions, so custom message and observation handling is only needed for custom turn control. The complete code is in a GitHub repository linked from the post. It follows two earlier posts on FunctionGemma fine-tuning and single-turn GRPO with Tunix.
What this means
The post shows a complete multi-turn, tool-using training loop on one TPU v6e-8 host, from environment definition through tool-call parsing to GRPO updates. The reported gain, 9.4% to 95% win rate in 100 steps, comes from a single game environment with a small action space (6x6 grid, one tool), so it demonstrates the pipeline rather than general agent capability. The handling of Gemma 4's native stop token is the practical detail: it lets one assistant turn span several tool exchanges, and the agent must track whether the model is awaiting a tool response. Builders who need less custom plumbing can start from the prebuilt ToolEnvironment, ToolAgent and ToolManager classes.
Related Articles
Google previews Tool Retrieval in Gemini Interactions API to load tool schemas only when needed
Google announced Tool Retrieval on November 19, 2026, available in preview in the Gemini Interactions API. Agents can keep large tool libraries on standby and load full parameter schemas only when the current task needs them.
Google makes Genie 4 available in Gemini API and AI Studio with 720p, 24 fps interactive worlds
Google made Genie 4 available through the Gemini API and Google AI Studio on November 19, 2026. The model generates continuous 720p video at 24 frames per second with synchronized native audio, and worlds respond to keyboard controls and live text prompts.
Google rolls out computer use in the Gemini app for Mac for multi-step desktop tasks
Google announced on November 19, 2026 that computer use is rolling out in the Gemini app for Mac. Gemini can carry out multi-step work across local files, desktop applications and websites, working in an isolated background window with a live preview in the chat.
Google builds isolated 'Mirror Dimension' web to train and test Chrome AI agents
Google's Chrome team built the Mirror Dimension, an isolated web ecosystem it fully controls, to train and test Gemini-based browsing agents without exposing them to the open internet. Google is detailing the project in a blog post, according to Wired.
Comments
Loading...