Distillation - Monte Platform

Distillation trains a student model to match a teacher model’s next-token distributions on the student’s own samples. It fits when a larger model already does the task well and a smaller model must serve it.

Prepare the Environment

Distillation trains on a training environment. The environment’s tasks supply the prompts. support-triage uses the same environment as GRPO:

monte env add --training support_escalation --as support-escalation-train

Start a Run

monte train support-triage \
  --config triage-distill \
  --data support-escalation-train \
  --steps 50 \
  --from-base

Parameters

Parameter	Description
`model`	The student
`distillation.teacher`	The teacher, such as `Qwen/Qwen3-32B`
`distillation.kl`	`reverse` makes the student focus on the teacher’s most likely outputs. `forward` makes it cover all of them.
`batch_size`	Prompts in one step
`max_seq_length`	The prompt plus the response, in tokens

The teacher and the student must share one tokenizer, so pick two models from one family in Models.