> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Distillation

> Train a smaller model from a larger teacher

Distillation trains a student model to match a teacher model's next-token distributions on the student's own samples. It fits when a larger model already does the task well and a smaller model must serve it.

## Prepare the Environment

Distillation trains on a [training environment](https://docs.trymonte.ai/environments/overview.md). The environment's tasks supply the prompts. `support-triage` uses the same environment as [GRPO](https://docs.trymonte.ai/training/grpo.md):

```bash theme={null}
monte env add --training support_escalation --as support-escalation-train
```

## Start a Run

```bash theme={null}
monte train support-triage \
  --config triage-distill \
  --data support-escalation-train \
  --steps 50 \
  --from-base
```

## Parameters

| Parameter | Description |
| - | - |
| `model` | The student |
| `distillation.teacher` | The teacher, such as `Qwen/Qwen3-32B` |
| `distillation.kl` | `reverse` makes the student focus on the teacher's most likely outputs. `forward` makes it cover all of them. |
| `batch_size` | Prompts in one step |
| `max_seq_length` | The prompt plus the response, in tokens |

The teacher and the student must share one tokenizer, so pick two models from one family in [Models](https://docs.trymonte.ai/training/models.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.