# CLI Source: https://docs.trymonte.ai/cli Install and use the Monte CLI Run every step of the platform from a terminal or a script. ## Get the CLI > [!NOTE] > > Monte is in closed beta. The CLI installs for teams with access. To request access, email [founders@trymonte.ai](mailto:founders@trymonte.ai). `monte login` opens a browser and stores a token on this machine. The same account signs in to the [Console](/platform#console). ```bash theme={null} curl -LsSf https://trymonte.ai/cli | sh monte login ``` ## Examples Create an experiment with a held-out eval and a development eval: ```bash theme={null} monte experiment create support-triage --held-out support-escalation --dev support-escalation-dev ``` Start an SFT run from the base model on a GPU instance: ```bash theme={null} monte train support-triage --recipe triage-sft --data support-replies --steps 200 --from-base --remote ``` Deploy the checkpoint that the run saved at its last step: ```bash theme={null} monte deployment create support-triage/train-07 --name support-triage ``` ## Command Map The pages follow the order of the work: set up, add data, measure, train, deploy, and inspect. | Step | Page | Commands | | - | - | - | | Set up | [Account and Cloud](/commands/setup) | `monte login`, `monte config`, `monte cloud connect`, `monte secret set`, `monte instance up` | | Add environments | [Environments](/commands/environments) | `monte env create`, `monte env add`, `monte env list`, `monte env show` | | Add data | [Datasets](/commands/datasets) | `monte dataset add`, `monte dataset generate`, `monte dataset list` | | Measure | [Experiments and Evals](/commands/evaluations) | `monte experiment create`, `monte experiment show`, `monte eval`, `monte eval rescore` | | Train | [Training and Checkpoints](/commands/training) | `monte train`, `monte train extend`, `monte checkpoint list`, `monte checkpoint download` | | Deploy | [Deployments and Keys](/commands/serving) | `monte deployment create`, `monte deployment update`, `monte deployment rollback`, `monte key create` | | Inspect | [Runs and Traces](/commands/observability) | `monte run list`, `monte run logs`, `monte trace list`, `monte compare` | [Conventions](/commands/setup#conventions) explains run and checkpoint addresses, shared flags, and exit codes. # Datasets Source: https://docs.trymonte.ai/commands/datasets Datasets, synthetic data, and anonymization These commands add, generate, and remove datasets. [Datasets](/datasets/overview) explains the row kinds and the overlap record. | Command | Description | | - | - | | [`monte dataset add`](#monte-dataset-add) | Add a dataset as a new version | | [`monte dataset generate`](#monte-dataset-generate) | Generate rows from seed examples | | [`monte dataset list`](#monte-dataset-list) | List datasets | | [`monte dataset show`](#monte-dataset-show) | Show one dataset and check its rows | | [`monte dataset delete`](#monte-dataset-delete) | Remove a dataset from artifact storage | ## monte dataset add ```text theme={null} monte dataset add --from-file --kind sft|preference [key flags] [anonymize flags] [--json] monte dataset add --from-hf [@] --file --kind sft|preference [key flags] [anonymize flags] [--json] ``` Add one JSONL file as a new dataset version. The command checks every row, checks the overlap with every eval environment ([The Overlap Record](/datasets/overview#the-overlap-record)), and pushes the rows to artifact storage. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | Lowercase letters, digits, and `-` | | `--from-file` | path | | A local JSONL file | | `--from-hf` | string | | A Hugging Face dataset repo. `@` pins a branch, tag, or commit. | | `--file` | path | required with `--from-hf` | The JSONL file inside the repo | | `--kind` | string | required | `sft` for demonstrations, `preference` for preference pairs | | `--input-key` | string | `input` | The prompt column of an `sft` row | | `--output-key` | string | `output` | The answer column of an `sft` row | | `--prompt-key` | string | `prompt` | The prompt column of a `preference` row | | `--chosen-key` | string | `chosen` | The preferred answer of a `preference` row | | `--rejected-key` | string | `rejected` | The rejected answer of a `preference` row | | `--anonymize` | flag | off | Replace personal details before the rows are stored. [Anonymization](/datasets/anonymization) explains it. | | `--entities` | list | every built-in type | With `--anonymize`: the types to find. NER types: `person`, `org`, `location`, `address`. Pattern types: `email`, `phone`, `ip`, `account`, `card`, `id` | | `--replace` | `placeholder`, `pseudonym`, or `redact` | `placeholder` | With `--anonymize`: what a found value becomes | | `--threshold` | number from 0 to 1 | `0.5` | With `--anonymize`: the lowest NER confidence that counts as a find | | `--keep` | path | none | With `--anonymize`: a text file of terms that are never replaced, one for each line | | `--patterns` | path | none | With `--anonymize`: a YAML file that maps a custom type to a regular expression | | `--preview` | integer | off | With `--anonymize`: print this many rows before and after, and store nothing | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte dataset add support-replies --from-file demos.jsonl --kind sft --anonymize ``` ```text theme={null} checked 6,480 rows (sft: messages) anonymized PERSON 11,070 EMAIL 1,788 PHONE 507 ORG 7,349 ACCOUNT 110 overlap none against support-escalation@1, support-escalation-dev@1 added support-replies@1 sha256 0be4a6f9... ``` | Error | Cause | | - | - | | `E_REFUSAL` | A row does not match the kind, or a row matches a held-out eval. The message names the line. | | `E_NOT_FOUND` | No file at the path, or no such repo, revision, or file on Hugging Face | ## monte dataset generate ```text theme={null} monte dataset generate --seeds --name --model [--kind sft|preference] [--rows ] [--turns ] [--topics ] [--dedup near|exact|off] [--judge ] [--grader ] [--min-score ] [--out ] [--remote] [--json] ``` Generate rows from seed examples, and drop the rows that fail the filters. The rows that pass go to a JSONL file for review, and the dropped rows go to a second file with a reason for each. The command does not add a dataset. [Synthetic Data](/datasets/synthetic-data) explains each step. The generator and the judge are open-weight models. Each is a Hugging Face ID, such as `Qwen/Qwen3-235B-A22B-Instruct-2507`, or a deployment name, such as `qwen3-235b`. | Flag | Type | Default | Description | | - | - | - | - | | `--seeds` | path | required | A JSONL file of seed rows in the format of `--kind`. With `--grader`, each seed also holds that environment's grader fields. | | `--kind` | string | `sft` | `sft` writes rows in the shape of the seeds: `input` and `output`, or a `messages` list with `tool_calls` when the grader's tasks declare tools. `preference` writes `prompt`, `chosen`, and `rejected` rows from preference seeds. | | `--name` | string | required | The name of the dataset to come. It also names the output files. | | `--rows` | integer | `500` | The number of rows to generate, before the filters | | `--model` | string | required | The generator: an open-weight model ID or a deployment name | | `--turns` | integer | `1` | The exchanges in each row. Above 1, each row is a conversation in the `messages` format. | | `--topics` | path | | A text file of topics, one on each line. Without it, the generator takes topics from the seeds. | | `--dedup` | string | `near` | `near` drops rows close to a seed or another row, `exact` drops only exact copies, `off` keeps every row | | `--judge` | string | | A judge that scores each row from 0 to 1: an open-weight model ID or a deployment name | | `--grader` | string | | A training environment whose grader scores each reply. An eval environment is refused. | | `--min-score` | number | `0.5` | The lowest judge or grader score that passes | | `--out` | path | `./.jsonl` | The output file. The dropped rows go to the same path with `.dropped.jsonl`. | | `--remote` | flag | off | Run the job on a GPU instance | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte dataset generate --seeds seeds.jsonl --name support-synth --rows 800 \ --model Qwen/Qwen3-235B-A22B-Instruct-2507 --judge zai-org/GLM-4.7 --min-score 0.7 \ --grader support-escalation-train --remote ``` ```text theme={null} generated 800 rows dropped 31 near copies, 42 below 0.7 on the judge, 18 below 0.7 on support-escalation-train@1 wrote ./support-synth.jsonl (709 rows), ./support-synth.dropped.jsonl (91 rows) ``` | Error | Cause | | - | - | | `E_REFUSAL` | A seed row does not match the format of `--kind`, or `--grader` names an eval environment | ## monte dataset list ```text theme={null} monte dataset list [--stored] [--json] ``` List the dataset versions on this machine. | Flag | Type | Default | Description | | - | - | - | - | | `--stored` | flag | off | List the datasets in artifact storage | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte dataset list ``` ```text theme={null} NAME KIND VERSION ROWS OVERLAP support-replies sft 1 6,480 none support-synth sft 1 709 support-escalation-dev@2 (3 rows) ``` ## monte dataset show ```text theme={null} monte dataset show [@] [--json] ``` Show one dataset: its kind, origin, row count, columns, anonymization counts, and overlap record. The command also checks the rows against their hash, and fetches the dataset from artifact storage if this machine does not hold it. | Flag | Type | Default | Description | | - | - | - | - | | `[@]` | argument | required | The dataset. A bare name means the newest version. | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte dataset show support-replies ``` ```text theme={null} dataset support-replies@1 (sft) rows 6,480 sha256 0be4a6f9... verified anonymized PERSON 11,070 EMAIL 1,788 PHONE 507 ORG 7,349 ACCOUNT 110 overlap none used by support-triage/train-07, support-triage/train-09 ``` ## monte dataset delete ```text theme={null} monte dataset delete [@] [--yes] [--json] ``` Remove a dataset from artifact storage and this machine. A bare name removes every version. Run history keeps the record that a run trained on the dataset, but not the rows. > [!WARNING] > > A delete does not remove what a model learned from the rows. [Delete a Dataset](/datasets/anonymization#delete-a-dataset) explains what else to remove. | Flag | Type | Default | Description | | - | - | - | - | | `[@]` | argument | required | The dataset, or one version of it | | `--yes` | flag | off | Skip the prompt | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte dataset delete support-replies ``` ```text theme={null} This removes 1 version of support-replies. 2 training runs used it: support-triage/train-07, support-triage/train-09. delete? [y/N] y Deleted support-replies@1 from artifact storage and this machine. ``` # Environments Source: https://docs.trymonte.ai/commands/environments Environments and their versions These commands create an environment, add it as a version, and move it between instances and artifact storage. [Environments](/environments/overview) explains the tasks, the server, and the grader of an environment. | Command | Description | | - | - | | [`monte env create`](#monte-env-create) | Create a new environment from a template | | [`monte env add`](#monte-env-add) | Add an eval or a training environment as a new version | | [`monte env list`](#monte-env-list) | List environments | | [`monte env show`](#monte-env-show) | Show one environment | | [`monte env install`](#monte-env-install) | Install an environment version on this machine | | [`monte env delete`](#monte-env-delete) | Remove an environment from artifact storage | ## monte env create ```text theme={null} monte env create [--json] ``` Write a new NeMo Gym resources server from the Gym template: `app.py` for the grader, a server config, and a `data/` directory. The files go under `monte-gym/resources_servers//`, or under the first directory in `MONTE_ENV_ROOTS`. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | Lowercase letters, digits, and `_`, for example `support_escalation` | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte env create support_escalation ``` ```text theme={null} Created monte-gym/resources_servers/support_escalation/ app.py configs/support_escalation.yaml data/ Write the verify method in app.py, then add it with monte env add. ``` | Error | Cause | | - | - | | `E_REFUSAL` | The directory exists, or the name has a character other than lowercase letters, digits, and `_` | ## monte env add ```text theme={null} monte env add --eval --as [--json] monte env add --training --as [--json] ``` Add an environment as an immutable version. The command runs Gym's prepare or collate step once, hashes the grader code, the task rows, and the configs, and writes a manifest and a fingerprint. A training environment's rows are checked against every eval, as [The Overlap Record](/datasets/overview#the-overlap-record) describes. Then the command pushes the result to artifact storage. Each add of a name makes a new version: `support-escalation@1`, then `support-escalation@2`. | Flag | Type | Default | Description | | - | - | - | - | | `--eval` | string | | An eval config under `monte-gym/benchmarks/`. The result is an eval environment. | | `--training` | string | | A resources server whose config declares a task file of `type: train`. The result is a training environment. | | `--as` | string | required | The environment name, with lowercase letters, digits, and `-` | | `--json` | flag | off | Print JSON on stdout | State `--eval` or `--training`, never both. ```bash theme={null} monte env add --eval support_escalation --as support-escalation ``` ```text theme={null} prepared benchmarks/support_escalation/prepare.py -> 200 tasks grader support_escalation sha256 7a3c1f88... tasks sha256 c90d4e12... added support-escalation@1 (eval) fingerprint 3f9a1c0e7b2d4a61 pushed gs://helpdesk-monte-artifacts/environments/support-escalation/1 ``` | Error | Cause | | - | - | | `E_REFUSAL` | The config declares no task file of the right type, a training environment names a closed-weight judge, or a training row matches a held-out eval | | `E_NOT_FOUND` | No eval config or resources server has that name | | `E_STORAGE` | Artifact storage is unreachable | ## monte env list ```text theme={null} monte env list [--stored] [--json] ``` List the environment versions on this machine. | Flag | Type | Default | Description | | - | - | - | - | | `--stored` | flag | off | List the environments in artifact storage | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte env list ``` ```text theme={null} NAME KIND VERSION TASKS FINGERPRINT support-escalation eval 1 200 3f9a1c0e7b2d4a61 support-escalation-dev eval 2 200 d18b2e70c4a95f13 support-escalation-train training 1 4,800 90c3b7e1a25d0f44 ``` ## monte env show ```text theme={null} monte env show [@] [--json] ``` Show one environment: its kind, its versions, its fingerprint, its task count, the hashes of its grader, its rows, and its config, and the experiments that pin it. | Flag | Type | Default | Description | | - | - | - | - | | `[@]` | argument | required | The environment. A bare name means the newest version. | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte env show support-escalation-dev ``` ```text theme={null} environment support-escalation-dev@2 (eval) versions 1, 2 fingerprint d18b2e70c4a95f13 tasks 200 sha256 51e7a0c3... (same as @1) config sha256 e41b09d2... (@1: 6c8f2a19...) grader support_escalation sha256 7a3c1f88... (same as @1) domain other pinned by support-triage-v2 ``` ## monte env install ```text theme={null} monte env install [@] | --all [--force] [--json] ``` Fetch an environment from artifact storage and install it on this machine. The command checks every hash against the manifest before it writes a file. | Flag | Type | Default | Description | | - | - | - | - | | `[@]` | argument | | The environment to install | | `--all` | flag | off | Install every environment in artifact storage | | `--force` | flag | off | Install again when the local copy is complete | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte env install support-escalation ``` ```text theme={null} support-escalation@1 verified 3 files against the manifest, installed in ~/monte-data/environments/support-escalation/1 ``` | Error | Cause | | - | - | | `E_CORRUPT_STATE` | A file fails its hash | | `E_NOT_FOUND` | Artifact storage holds no environment with that name | ## monte env delete ```text theme={null} monte env delete [@] [--yes] [--json] ``` Remove an environment from artifact storage and from this machine. A bare name removes every version. | Flag | Type | Default | Description | | - | - | - | - | | `[@]` | argument | required | The environment, or one version of it | | `--yes` | flag | off | Skip the prompt | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte env delete support-escalation-dev@1 ``` ```text theme={null} E_IN_USE: support-escalation-dev@1 is pinned by experiment support-triage (delete the experiment first, or keep the version) ``` | Error | Cause | | - | - | | `E_IN_USE` | An experiment pins the version | A training environment is not pinned by an experiment, so its delete proceeds. Run history keeps the record that a run trained on it. # Experiments and Evals Source: https://docs.trymonte.ai/commands/evaluations Experiments, evals, baselines, and rescores These commands fix the evals that runs are compared on, and score models on them. [Experiments](/evaluation/experiments) explains held-out and development evals. [Scoring Models](/evaluation/scoring-models) explains the baseline and the change. | Command | Description | | - | - | | [`monte experiment create`](#monte-experiment-create) | Create an experiment | | [`monte experiment list`](#monte-experiment-list) | List experiments | | [`monte experiment show`](#monte-experiment-show) | Show the evals, baselines, and newest differences of an experiment | | [`monte experiment baselines`](#monte-experiment-baselines) | Show which evals have a baseline | | [`monte experiment delete`](#monte-experiment-delete) | Remove an experiment | | [`monte experiment sync`](#monte-experiment-sync) | Copy run history from artifact storage | | [`monte experiment push`](#monte-experiment-push) | Copy run files to artifact storage | | [`monte eval`](#monte-eval) | Score a model or a checkpoint on one eval | | [`monte eval rescore`](#monte-eval-rescore) | Score the replies of an eval run again with another grader | ## monte experiment create ```text theme={null} monte experiment create --held-out [--dev ] [--description ] [--json] ``` Create an experiment. The command pins each eval by fingerprint with its own eval settings. Nothing in the experiment changes after that. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | Lowercase letters, digits, and `-` | | `--held-out` | list | required | The held-out evals, comma-separated. Each is an eval environment, as `` or `@`. | | `--dev` | list | none | The development evals, comma-separated | | `--description` | string | none | Free text for the record | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment create support-triage --held-out support-escalation --dev support-escalation-dev \ --description "Escalate or resolve support tickets" ``` ```text theme={null} experiment 'support-triage' created (1 held-out, 1 dev) held-out support-escalation@1 fingerprint 3f9a1c0e7b2d4a61 dev support-escalation-dev@1 fingerprint a04e9d2c5f1b8e37 ``` | Error | Cause | | - | - | | `E_EXPERIMENT_EXISTS` | An experiment has that name | | `E_NOT_FOUND` | A name is not an eval environment in artifact storage | | `E_REFUSAL` | A name is a training environment, or a held-out eval matches the rows of a training environment or a dataset ([The Overlap Record](/datasets/overview#the-overlap-record)) | ## monte experiment list ```text theme={null} monte experiment list [--remote] [--json] ``` List experiments with owner, creation time, eval count, and run count. | Flag | Type | Default | Description | | - | - | - | - | | `--remote` | flag | off | List the experiments in the control plane, not only those on this machine. | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment list ``` ```text theme={null} NAME OWNER CREATED EVALS RUNS support-triage lee@example.com 2026-10-14 1 held-out, 1 dev 9 ``` ## monte experiment show ```text theme={null} monte experiment show [--json] ``` Show the evals of an experiment, its base models, how many checkpoints were scored on each held-out eval, and one line for each eval with a baseline: the baseline score, the newest score, and the paired mean difference of task scores. [Read the Result](/evaluation/scoring-models#read-the-result) explains the line. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment show support-triage ``` ```text theme={null} evals held-out support-escalation@1 dev support-escalation-dev@1 models Qwen/Qwen3-8B @ a09a3545 held-out support-escalation 2 checkpoints scored support-escalation-dev: 0.50 -> 0.60 (delta +0.10 ± 0.06 paired, 30 fixed / 10 broken, eval@200) support-escalation: 0.49 -> 0.61 (delta +0.12 ± 0.05 paired, 28 fixed / 4 broken, eval@300) ``` ## monte experiment baselines ```text theme={null} monte experiment baselines [--json] ``` Show which evals have a baseline for each base model. `monte train` refuses until every eval has one. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment baselines support-triage ``` ```text theme={null} model Qwen/Qwen3-8B @ a09a3545 support-escalation-dev dev eval-01 0.50 support-escalation held-out eval-02 0.49 ready to train ``` ## monte experiment delete ```text theme={null} monte experiment delete --confirm [--json] ``` Remove an experiment, its run history on this machine and in artifact storage, and the weights of its checkpoints. Environment and dataset versions stay in artifact storage. > [!WARNING] > > A delete cannot be undone. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `--confirm` | string | required | The experiment name, typed in full | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment delete support-triage-v2 --confirm support-triage-v2 ``` ```text theme={null} Deleted experiment support-triage-v2: 3 runs, 0 checkpoints. ``` | Error | Cause | | - | - | | `E_IN_USE` | A deployment serves a checkpoint of the experiment | ## monte experiment sync ```text theme={null} monte experiment sync [...] [--json] ``` Copy run history from artifact storage to this machine. It never deletes a local file. | Flag | Type | Default | Description | | - | - | - | - | | `...` | arguments | all experiments | The experiments to copy | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment sync support-triage ``` ```text theme={null} support-triage 4 new runs, 2 updated, 812 files (38 MB) ``` ## monte experiment push ```text theme={null} monte experiment push [--run ] [--json] ``` Copy the run files of one experiment (traces, per-step metrics, and logs) to artifact storage. Run metadata reaches the control plane on its own, and the Console reads it there. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `--run` | string | all runs | Push only one run | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte experiment push support-triage --run eval-03 ``` ```text theme={null} support-triage/eval-03 pushed 204 files (9.1 MB) ``` ## monte eval ```text theme={null} monte eval [] [--model ] [--checkpoint / | --step ] [--gpu ] [--gpus ] [--remote | --push] [--smoke] [--dry-run] [--yes] [--json] ``` Score one model on one eval of an experiment, and record an eval run. With no `--model`, `--checkpoint`, or `--step`, the target is the newest checkpoint. The eval settings come from the experiment, and no flag changes them. A checkpoint on a held-out eval asks for confirmation and prints how many checkpoints were scored on it before. The experiment records the count, and `monte experiment show` prints it. `--yes` skips the prompt. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `` | argument | the only development eval | The eval to score. A held-out eval is never the default. | | `--model` | string | none | Score this base model, as a Hugging Face model ID pinned by its snapshot revision. Required when the experiment has no checkpoint. | | `--checkpoint` | string | the newest checkpoint | One exact checkpoint, as `/` | | `--step` | integer | | The checkpoint at one absolute step. Refuses when two runs saved a checkpoint at that step. | | `--gpu` | string | the smallest type that fits the model | `l4`, `l40s`, `a100-80gb`, `h100`, or `h200`. With `--remote` only. | | `--gpus` | integer | the smallest count that fits the model | The number of GPUs, with `--remote` only. A count above 8 spans several nodes. | | `--remote` | flag | off | Submit the eval to a GPU instance and return | | `--push` | flag | off | Run here and upload the results after each state change | | `--smoke` | flag | off | Score the first 3 tasks. A smoke run never counts as a result or a baseline. | | `--dry-run` | flag | off | Print the target, the eval, and the settings, and start nothing | | `--yes` | flag | off | Skip the `launch?` prompt | | `--json` | flag | off | Print JSON on stdout | `--model`, `--checkpoint`, and `--step` exclude each other. `--remote` does not combine with `--push` or `--dry-run`. ```bash theme={null} monte eval support-triage --checkpoint train-07/200 --remote ``` ```text theme={null} run support-triage/eval-03 (eval) eval support-escalation-dev@1 (dev, 200 tasks) target checkpoint train-07/200 on Qwen/Qwen3-8B @ a09a3545 gpus h100 x1 launch? [y/N] y Submitted support-triage/eval-03 to gpu-4f2a. Follow it with: monte run logs support-triage/eval-03 -f ``` | Error | Cause | | - | - | | `E_REFUSAL` | No `` and more than one development eval, no `--model` and no checkpoint, or `--step` matches two checkpoints | | `E_PREFLIGHT` | The eval changed its fingerprint, or the run lock is held | | `E_NOT_FOUND` | No such experiment, eval, or checkpoint | ## monte eval rescore ```text theme={null} monte eval rescore / --into [--eval ] [--remote | --push] [--dry-run] [--yes] [--json] ``` Score the replies of an eval run again with the grader of another experiment's eval, and record a new eval run there. No model is served: the command reads the original run's traces. The new run counts as a result in the target experiment, and a rescored baseline is a baseline there. [Fix a Grader and Rescore](/environments/graders#fix-a-grader-and-rescore) shows the full sequence. | Flag | Type | Default | Description | | - | - | - | - | | `/` | argument | required | The original eval run | | `--into` | string | required | The target experiment. It cannot be the experiment of the original run. | | `--eval` | string | the one eval whose tasks match | The eval in the target experiment, when more than one has the same tasks | | `--remote` | flag | off | Run the grader on a GPU instance and return, for a judge model that needs a GPU | | `--push` | flag | off | Run here and upload the results after each state change | | `--dry-run` | flag | off | Print the original run, the target eval, and the task match, and start nothing | | `--yes` | flag | off | Skip the `launch?` prompt | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte eval rescore support-triage/eval-03 --into support-triage-v2 ``` ```text theme={null} tasks 200 of 200 match by task hash settings match launch? [y/N] y support-triage-v2/eval-03 succeeded support-escalation-dev=0.57±0.07 (200 tasks, 0 served) ``` | Error | Cause | | - | - | | `E_NOT_COMPARABLE` | The target eval has other tasks, or other eval settings, than the original eval | | `E_REFUSAL` | The original run is not an eval run, is a smoke run, or has no stored traces | # Runs and Traces Source: https://docs.trymonte.ai/commands/observability Runs, traces, and run comparison These commands read what runs recorded: rows, events, traces, and the difference between two runs. [Metrics](/evaluation/metrics) explains the output. | Command | Description | | - | - | | [`monte run list`](#monte-run-list) | List the runs of an experiment | | [`monte run show`](#monte-run-show) | Show one run in full | | [`monte run logs`](#monte-run-logs) | Print or follow the events of a run | | [`monte run stop`](#monte-run-stop) | Stop a running run | | [`monte trace list`](#monte-trace-list) | List the traces of a run | | [`monte trace download`](#monte-trace-download) | Write the traces of a run to a file | | [`monte compare`](#monte-compare) | Compare two eval runs task by task | ## monte run list ```text theme={null} monte run list [--kind eval|train] [--status ] [--all] [--json] ``` List the runs of an experiment, oldest first, with the alias of each succeeded run (`eval@200`), its status, and its score as `=±`. | Flag | Type | Default | Description | | - | - | - | - | | `` | argument | required | The experiment | | `--kind` | string | both | `eval` or `train` | | `--status` | string | all | `running`, `succeeded`, `failed`, or `abandoned` | | `--all` | flag | off | Include smoke runs, marked `[not evidence]` | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte run list support-triage ``` ```text theme={null} RUN ALIAS STATUS DETAIL SCORE eval-01 eval@0 succeeded support-escalation-dev base support-escalation-dev=0.50±0.07 eval-02 eval@0 succeeded support-escalation base support-escalation=0.49±0.07 ... train-07 train@200 succeeded triage-sft support-replies from base eval-03 eval@200 succeeded support-escalation-dev train-07/200 support-escalation-dev=0.60±0.07 train-09 - running triage-sft support-replies from train-07/200 ``` ## monte run show ```text theme={null} monte run show / [--metrics ] [--json] ``` Show one run in full: status, target, settings, scores, eval checks, and provenance. A training run also shows its recipe, training data version, and checkpoint. | Flag | Type | Default | Description | | - | - | - | - | | `/` | argument | required | The run, by ID or alias | | `--metrics` | string | all | With `--json`, include the per-step series for these comma-separated metric tags | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte run show support-triage/eval-03 ``` ```text theme={null} run support-triage/eval-03 (eval) succeeded target train-07/200 on Qwen/Qwen3-8B @ a09a3545 score 0.60 ± 0.07 vs baseline +0.10 ± 0.06 paired (30 fixed / 10 broken, eval-01) checks mean_output_tokens 142 parse_fail_rate 0.00 truncations 0.01 repetition 0.02 ``` ## monte run logs ```text theme={null} monte run logs / [-f] [--tail ] ``` Print the events of a run as JSON lines: `progress`, `checkpoint_saved`, and `score_recorded`, and a final `status` event. | Flag | Type | Default | Description | | - | - | - | - | | `/` | argument | required | The run | | `-f` | flag | off | Keep the output open and print new events until the run ends | | `--tail` | integer | all | Print only the last `n` events first | ```bash theme={null} monte run logs support-triage/train-07 -f --tail 2 ``` ```text theme={null} {"event": "progress", "step": 200, "loss": 0.407, "step_time_s": 11.8} {"event": "checkpoint_saved", "checkpoint": "train-07/200", "kind": "both"} {"event": "status", "status": "succeeded"} ``` ## monte run stop ```text theme={null} monte run stop / [--json] ``` Stop a running run. Its status becomes `abandoned`. The GPU instance is released. Saved checkpoints and recorded scores stay. | Flag | Type | Default | Description | | - | - | - | - | | `/` | argument | required | The run | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte run stop support-triage/train-09 ``` ```text theme={null} Stopped support-triage/train-09 at step 264 of 300. Status abandoned. Last checkpoint train-09/250. ``` | Error | Cause | | - | - | | `E_NOT_RUNNING` | The run is not running | ## monte trace list ```text theme={null} monte trace list / [--eval ] [--failed] [--step ] [--sort task|score] [--json] ``` List the traces of a run, one row for each task, with the score, the parsed and expected answers, the tokens, and the stop reason. A training run with a grader has one trace for each scored rollout. | Flag | Type | Default | Description | | - | - | - | - | | `/` | argument | required | The run | | `--eval` | string | the eval that the run scored | The eval, for a run with traces on more than one | | `--failed` | flag | off | Only the traces with a score below `1.0` | | `--step` | integer | all steps | For a training run, only the rollouts of one step | | `--sort` | string | `task` | `task` or `score` | | `--json` | flag | off | Print JSON on stdout | ```bash theme={null} monte trace list support-triage/eval-03 --failed --sort score ``` ```text theme={null} TASK SCORE EXTRACTED EXPECTED TOKENS STOP t-004 0.00 resolve escalate: refund 412/96 - t-031 0.00 escalate: complaint escalate: contract 388/41 - t-142 0.00 - resolve 520/2048 length 80 failed of 200 ``` ## monte trace download ```text theme={null} monte trace download / [--eval ] [--step ] [-o ] [--json] monte trace download --deployment [--since