GRPO - Monte Platform

Group relative policy optimization samples several responses for each prompt, scores them with the environment’s grader, and moves the model toward the responses that score above their group’s mean. It fits when a grader, in code or with an open-weight judge, can score an output, including on multi-turn and tool-use tasks.

Prepare the Environment

GRPO trains on a training environment: a set of tasks and a grader.

monte env add --training support_escalation --as support-escalation-train

The support_escalation grader rewards an escalation with the right category, or a resolution with the right facts from the knowledge base. The starting model must earn a reward on some tasks. When it scores near 0, run SFT first.

Start a Run

monte train support-triage \
  --config triage-grpo \
  --data support-escalation-train \
  --steps 50 \
  --parent train-07/200

Parameters

Parameter	Description
`batch_size`	Prompts in one step
`grpo.group_size`	Samples for each prompt
`grpo.max_turns`	Turns in one rollout, for a multi-turn or tool-use task
`grpo.temperature`	The sampling temperature
`max_seq_length`	The full rollout in tokens, with every tool call and result