> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments

> Fixed evals that every run is compared on

An experiment is a fixed set of eval environments with pinned settings. Every run in it is scored on the same tasks under the same rules, so two scores differ only because the model differs.

## Eval Roles

| Role | Scored | Use |
| - | - | - |
| Development | On any checkpoint, as often as needed | Choose the next training run |
| Held-out | At the baseline, then once on the chosen checkpoint | The result to report |

Choose the checkpoint on the development evals, then score the held-out eval on that checkpoint only. An experiment can have any number of development evals.

## Create an Experiment

```bash theme={null}
monte experiment create support-triage \
  --held-out support-escalation \
  --dev support-escalation-dev
```

Both flags take a comma-separated list of eval environments ([Environments](https://docs.trymonte.ai/environments/overview.md#create-and-add-an-environment)). `--held-out` is required. Monte refuses a held-out eval whose tasks overlap the task rows of a training environment.

The experiment pins each eval by its fingerprint, the eval settings, and the roles. Monte checks every fingerprint before each run and refuses the run if an eval changed. A changed eval needs a new experiment.

## Take the Baselines

Score the base model on every eval before the first training run:

```bash theme={null}
monte eval support-triage support-escalation-dev --model Qwen/Qwen3-8B
monte eval support-triage support-escalation --model Qwen/Qwen3-8B
monte experiment baselines support-triage
```

Every later score is a change from this baseline. `monte train` refuses to start until each eval has a baseline for its base model. A second base model needs its own baselines.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.