Traces Into Datasets
Reviewed traces go into a new version of a dataset. A corrected reply is a demonstration row for SFT, and a reviewer’s edit with the model’s original reply is a pair for DPO. Add the earlier rows and the new ones together, and the dataset gets a new version, such assupport-replies@2. Each run records the exact version it trained on.
Traces Into Environments
A trace’s request becomes a task row in the training environment. The task takes the trace’s messages and tools in the task format. For a conversation over several requests, join its traces by a session ID inmetadata. The reviewer fills in the grader fields, such as should_escalate and expected_category. Adding the task file again makes a new version, such as support-escalation-train@2, for a GRPO run:
Each Round
Train from the deployed checkpoint with--parent, on the new versions. Pick the checkpoint on the development eval, and compare it task by task with the deployed checkpoint. Score the held-out eval once for each round. Its change from the baseline compares directly with every earlier round, but a held-out eval scored over many rounds starts to overstate gains, so refresh it with new traffic in a new experiment every few rounds. Then move traffic in steps, and the deployment keeps collecting traces for the next round. Every step is a CLI command, so a team can run a round from a script on its own schedule.
New Kinds of Requests
When traffic shifts, pinned evals no longer cover it. Add the new requests to an eval environment as a new version, and create a new experiment with it. Take its baselines before the next run. Eachmonte dataset add and monte env add --training checks the new rows against every eval, so a trace that matches an eval task does not leak into training unnoticed.
