Skip to main content
Every run records metrics. A training run records the trainer’s metrics at each step, and an eval run records its score and checks on the responses. The Console charts both, and monte run show prints them in the terminal.

​
Training Metrics

The trainer writes its metrics at every step, grouped into families by prefix. The exact set depends on the method and the trainer.

​
Eval Metrics

The checks explain a score change. For example, a rise in parse_fail_rate with a fall in score shows a format problem, not a quality problem.

​
View Metrics

In the Console, click Training, select the experiment and the run, then click Metrics. Each series has a chart that marks the checkpoint steps. Click the star on a chart to pin it for every run. In the terminal:
--json with --metrics prints each series as a list of {"step": n, "value": x} points. monte run logs prints the run’s events as JSON lines.

​
Compare Runs

monte compare takes two eval runs on the same eval. It shows the paired difference and each task that one run passed and the other failed. The first run is the reference.
Both runs must be on the same eval, in the same experiment, and from the same base model, and neither can be a smoke run. Otherwise the command refuses and names the condition. To compare in the Console:
  1. Click Evaluations and select the experiment.
  2. Select two eval runs on the same eval.
  3. Click Compare.
The view lists each task as Fixed, Broken, or Same. Click a row to open the two traces side by side.