These commands fix the evals that runs are compared on, and score models on them. Experiments explains held-out and development evals. Scoring Models explains the baseline and the change.
Create an experiment. The command pins each eval by fingerprint with its own eval settings. Nothing in the experiment changes after that.
Flag
Type
Default
Description
<name>
argument
required
Lowercase letters, digits, and -
--held-out
list
required
The held-out evals, comma-separated. Each is an eval environment, as <name> or <name>@<version>.
--dev
list
none
The development evals, comma-separated
--description
string
none
Free text for the record
--json
flag
off
Print JSON on stdout
monte experiment create support-triage --held-out support-escalation --dev support-escalation-dev \ --description "Escalate or resolve support tickets"
experiment 'support-triage' created (1 held-out, 1 dev) held-out support-escalation@1 fingerprint 3f9a1c0e7b2d4a61 dev support-escalation-dev@1 fingerprint a04e9d2c5f1b8e37
Error
Cause
E_EXPERIMENT_EXISTS
An experiment has that name
E_NOT_FOUND
A name is not an eval environment in artifact storage
E_REFUSAL
A name is a training environment, or a held-out eval overlaps the task rows of a training environment
Show the evals of an experiment, its base models, and one line for each eval with a baseline: the baseline score, the newest score, and the paired mean difference of task scores. Read the Result explains the line.
monte experiment delete <name> --confirm <name> [--json]
Remove an experiment, its run history on this machine and in artifact storage, and the weights of its checkpoints. Environment and dataset versions stay in artifact storage.
A delete cannot be undone.
Flag
Type
Default
Description
<name>
argument
required
The experiment
--confirm
string
required
The experiment name, typed in full
--json
flag
off
Print JSON on stdout
monte experiment delete support-triage-v2 --confirm support-triage-v2
Score one model on one eval of an experiment, and record an eval run. With no --model, --checkpoint, or --step, the target is the newest checkpoint. The eval settings come from the experiment, and no flag changes them.
Flag
Type
Default
Description
<experiment>
argument
required
The experiment
<eval>
argument
the only development eval
The eval to score. A held-out eval is never the default.
--model
string
none
Score this base model, as a Hugging Face model ID pinned by its snapshot revision. Required when the experiment has no checkpoint.
--checkpoint
string
the newest checkpoint
One exact checkpoint, as <run>/<step>
--step
integer
The checkpoint at one absolute step. Refuses when two runs saved a checkpoint at that step.
--gpu
string
the smallest type that fits the model
l4, l40s, a100-80gb, h100, or h200. With --remote only.
--gpus
integer
the smallest count that fits the model
The number of GPUs, with --remote only. A count above 8 spans several nodes.
--remote
flag
off
Submit the eval to a GPU instance and return
--push
flag
off
Run here and upload the results after each state change
--smoke
flag
off
Score the first 3 tasks. A smoke run never counts as a result or a baseline.
--dry-run
flag
off
Print the target, the eval, and the settings, and start nothing
--yes
flag
off
Skip the launch? prompt
--json
flag
off
Print JSON on stdout
--model, --checkpoint, and --step exclude each other. --remote does not combine with --push or --dry-run.
monte eval support-triage --checkpoint train-07/200 --remote
run support-triage/eval-03 (eval)eval support-escalation-dev@1 (dev, 200 tasks)target checkpoint train-07/200 on Qwen/Qwen3-8B @ a09a3545gpus h100 x1launch? [y/N] ySubmitted support-triage/eval-03 to gpu-4f2a. Follow it with: monte run logs support-triage/eval-03 -f
Error
Cause
E_REFUSAL
No <eval> and more than one development eval, no --model and no checkpoint, or --step matches two checkpoints
E_PREFLIGHT
The eval changed its fingerprint, or the run lock is held
Score the replies of an eval run again with the grader of another experiment’s eval, and record a new eval run there. No model is served: the command reads the original run’s traces. The new run counts as a result in the target experiment, and a regraded baseline is a baseline there. Fix a Grader and Regrade shows the full sequence.
Flag
Type
Default
Description
<experiment>/<run>
argument
required
The original eval run
--into
string
required
The target experiment. It cannot be the experiment of the original run.
--eval
string
the one eval whose tasks match
The eval in the target experiment, when more than one has the same tasks
--remote
flag
off
Run the grader on a GPU instance and return, for a judge model that needs a GPU
--push
flag
off
Run here and upload the results after each state change
--dry-run
flag
off
Print the original run, the target eval, and the task match, and start nothing
--yes
flag
off
Skip the launch? prompt
--json
flag
off
Print JSON on stdout
monte grade support-triage/eval-03 --into support-triage-v2
tasks 200 of 200 match by task hash settings matchlaunch? [y/N] ysupport-triage-v2/eval-03 succeeded support-escalation-dev=0.57±0.07 (200 tasks, 0 served)
Error
Cause
E_NOT_COMPARABLE
The target eval has other tasks, or other eval settings, than the original eval
E_REFUSAL
The original run is not an eval run, is a smoke run, or has no stored traces
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Monte Platform","colors":{"primary":"#315594","light":"#70A6C1","dark":"#2F3675"},"logo":{"light":"https://mintcdn.com/monte-inc/gDE0uEcoDx7_moLr/logo/light.png?fit=max\u0026auto=format\u0026n=gDE0uEcoDx7_moLr\u0026q=85\u0026s=fe282530810a529d346a3d34b18dfa1e","dark":"https://mintcdn.com/monte-inc/gDE0uEcoDx7_moLr/logo/dark.png?fit=max\u0026auto=format\u0026n=gDE0uEcoDx7_moLr\u0026q=85\u0026s=e6807ec55b887d13310c530d0c73cf5e"},"favicon":"https://mintcdn.com/monte-inc/gDE0uEcoDx7_moLr/favicon.png?fit=max\u0026auto=format\u0026n=gDE0uEcoDx7_moLr\u0026q=85\u0026s=1d58f7325198c466585e820513d03b27","contextual":{"options":["copy","view","chatgpt","claude","perplexity","mcp","cursor","vscode"],"display":"header"},"styling":{"latex":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"monte-inc","hasAudioNarration":false,"actualSubdomain":"monte-inc","customDomains":["docs.trymonte.ai"],"gitSource":{"type":"github","owner":"monte-inc","repo":"monte-platform-docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"chroma":{"latestBuiltDeploymentHistoryId":"6ae506bdefaa129e5fed20c2","collectionId":"cf4218b7-47f4-42be-bf41-dd92eb3604b0","liveDeploymentHistoryId":"6ae506bdefaa129e5fed20c2","schemaVersion":"v3"},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6ae506bdefaa129e5fed20c2","clientVersion":"0.0.4044","searchFilterCounts":[{"language":"default","tag":"Platform","count":28},{"language":"default","tag":"CLI","count":8},{"language":"default","tag":"Commands","count":7},{"language":"default","tag":"Training","count":7},{"language":"default","tag":"Get Started","count":4},{"language":"default","tag":"Inference","count":4},{"language":"default","tag":"Environments","count":4},{"language":"default","tag":"Methods","count":4},{"language":"default","tag":"Datasets","count":3},{"language":"default","tag":"Evaluation","count":3}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"byok-sep-25","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"vite-origin","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-updated-9-15","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"style.css","content":".mermaid,[class*=mermaid]{text-align:center}.mermaid>svg,[class*=mermaid]>svg{display:block;margin-inline:auto}.mermaid .cluster text,.mermaid .cluster-label p,.mermaid .cluster-label span,.mermaid .cluster-label text{fill:#17181b;color:#17181b}.dark .mermaid .cluster text,.dark .mermaid .cluster-label p,.dark .mermaid .cluster-label span,.dark .mermaid .cluster-label text{fill:#f0f1f2;color:#f0f1f2}"}],"page":{"pagePath":"commands/evaluations","slug":"/commands/evaluations","pageMetadata":{"title":"Evaluations","description":"Experiments, evals, baselines, and regrades","href":"/commands/evaluations"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Experiments, evals, baselines, and regrades\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,pre:`pre`,span:`span`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{CodeBlock,Heading,Table,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`These commands fix the evals that runs are compared on, and score models on them. `,_jsx(_components.a,{href:`/evaluation/experiments`,children:`Experiments`}),` explains held-out and development evals. `,_jsx(_components.a,{href:`/evaluation/scoring-models`,children:`Scoring Models`}),` explains the baseline and the change.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Command`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-create`,children:_jsx(_components.code,{children:`monte experiment create`})})}),_jsx(_components.td,{children:`Create an experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-list`,children:_jsx(_components.code,{children:`monte experiment list`})})}),_jsx(_components.td,{children:`List experiments`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-show`,children:_jsx(_components.code,{children:`monte experiment show`})})}),_jsx(_components.td,{children:`Show the evals, baselines, and newest differences of an experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-baselines`,children:_jsx(_components.code,{children:`monte experiment baselines`})})}),_jsx(_components.td,{children:`Show which evals have a baseline`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-delete`,children:_jsx(_components.code,{children:`monte experiment delete`})})}),_jsx(_components.td,{children:`Remove an experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-sync`,children:_jsx(_components.code,{children:`monte experiment sync`})})}),_jsx(_components.td,{children:`Copy run history from artifact storage`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-experiment-push`,children:_jsx(_components.code,{children:`monte experiment push`})})}),_jsx(_components.td,{children:`Copy run history to artifact storage`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-eval`,children:_jsx(_components.code,{children:`monte eval`})})}),_jsx(_components.td,{children:`Score a model or a checkpoint on one eval`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#monte-grade`,children:_jsx(_components.code,{children:`monte grade`})})}),_jsx(_components.td,{children:`Score the replies of an eval run again with another grader`})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-create`,children:`monte experiment create`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment create \u003cname> --held-out \u003cevals> [--dev \u003cevals>] [--description \u003ctext>] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`Create an experiment. The command pins each eval by fingerprint with its own eval settings. Nothing in the experiment changes after that.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsxs(_components.td,{children:[`Lowercase letters, digits, and `,_jsx(_components.code,{children:`-`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--held-out`})}),_jsx(_components.td,{children:`list`}),_jsx(_components.td,{children:`required`}),_jsxs(_components.td,{children:[`The held-out evals, comma-separated. Each is an eval environment, as `,_jsx(_components.code,{children:`\u003cname>`}),` or `,_jsx(_components.code,{children:`\u003cname>@\u003cversion>`}),`.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--dev`})}),_jsx(_components.td,{children:`list`}),_jsx(_components.td,{children:`none`}),_jsx(_components.td,{children:`The development evals, comma-separated`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--description`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`none`}),_jsx(_components.td,{children:`Free text for the record`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` create`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --held-out`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-escalation`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --dev`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-escalation-dev`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --description`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Escalate or resolve support tickets\"`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`4`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`4`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`experiment 'support-triage' created (1 held-out, 1 dev)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` held-out support-escalation@1 fingerprint 3f9a1c0e7b2d4a61`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` dev support-escalation-dev@1 fingerprint a04e9d2c5f1b8e37`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Error`}),_jsx(_components.th,{children:`Cause`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_EXPERIMENT_EXISTS`})}),_jsx(_components.td,{children:`An experiment has that name`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_NOT_FOUND`})}),_jsx(_components.td,{children:`A name is not an eval environment in artifact storage`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_REFUSAL`})}),_jsx(_components.td,{children:`A name is a training environment, or a held-out eval overlaps the task rows of a training environment`})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-list`,children:`monte experiment list`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment list [--remote] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`List experiments with owner, creation time, eval count, and run count.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--remote`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Read artifact storage instead of this machine`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` list`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`NAME OWNER CREATED EVALS RUNS`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`support-triage [email protected] 2026-10-14 1 held-out, 1 dev 9`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-show`,children:`monte experiment show`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment show \u003cname> [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Show the evals of an experiment, its base models, and one line for each eval with a baseline: the baseline score, the newest score, and the paired mean difference of task scores. `,_jsx(_components.a,{href:`/evaluation/scoring-models#read-the-result`,children:`Read the Result`}),` explains the line.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` show`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`6`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`evals held-out support-escalation@1 dev support-escalation-dev@1`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`models Qwen/Qwen3-8B @ a09a3545`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` support-escalation-dev: 0.50 -> 0.60 (delta +0.10 ± 0.06 paired, 30 fixed / 10 broken, eval@200)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` support-escalation: 0.49 -> 0.58 (delta +0.09 ± 0.05 paired, 24 fixed / 6 broken, eval@200)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-baselines`,children:`monte experiment baselines`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment baselines \u003cname> [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Show which evals have a baseline for each base model. `,_jsx(_components.code,{children:`monte train`}),` refuses until every eval has one.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` baselines`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`model Qwen/Qwen3-8B @ a09a3545`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` support-escalation-dev dev eval-01 0.50`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` support-escalation held-out eval-02 0.49`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`ready to train`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-delete`,children:`monte experiment delete`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment delete \u003cname> --confirm \u003cname> [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`Remove an experiment, its run history on this machine and in artifact storage, and the weights of its checkpoints. Environment and dataset versions stay in artifact storage.`}),`\n`,_jsx(Warning,{children:_jsx(_components.p,{children:`A delete cannot be undone.`})}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--confirm`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment name, typed in full`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` delete`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage-v2`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --confirm`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage-v2`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Deleted experiment support-triage-v2: 3 runs, 0 checkpoints.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Error`}),_jsx(_components.th,{children:`Cause`})]})}),_jsx(_components.tbody,{children:_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_IN_USE`})}),_jsx(_components.td,{children:`A deployment serves a checkpoint of the experiment`})]})})]}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-sync`,children:`monte experiment sync`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment sync [\u003cname>...] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`Copy run history from artifact storage to this machine. It never deletes a local file.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>...`})}),_jsx(_components.td,{children:`arguments`}),_jsx(_components.td,{children:`all experiments`}),_jsx(_components.td,{children:`The experiments to copy`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` sync`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`support-triage 4 new runs, 2 updated, 812 files (38 MB)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`2`,id:`monte-experiment-push`,children:`monte experiment push`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte experiment push \u003cname> [--run \u003crun>] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`Copy the run history of one experiment to artifact storage, which the Console reads.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cname>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--run`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`all runs`}),_jsx(_components.td,{children:`Push only one run`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` experiment`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` push`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --run`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` eval-03`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`support-triage/eval-03 pushed 204 files (9.1 MB)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`2`,id:`monte-eval`,children:`monte eval`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte eval \u003cexperiment> [\u003ceval>] [--model \u003chf-id>] [--checkpoint \u003crun>/\u003cstep> | --step \u003cn>]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` [--gpu \u003ctype>] [--gpus \u003cn>] [--remote | --push] [--smoke] [--dry-run] [--yes] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Score one model on one eval of an experiment, and record an eval run. With no `,_jsx(_components.code,{children:`--model`}),`, `,_jsx(_components.code,{children:`--checkpoint`}),`, or `,_jsx(_components.code,{children:`--step`}),`, the target is the newest checkpoint. The eval settings come from the experiment, and no flag changes them.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cexperiment>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The experiment`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003ceval>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`the only development eval`}),_jsx(_components.td,{children:`The eval to score. A held-out eval is never the default.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--model`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`none`}),_jsx(_components.td,{children:`Score this base model, as a Hugging Face model ID pinned by its snapshot revision. Required when the experiment has no checkpoint.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--checkpoint`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`the newest checkpoint`}),_jsxs(_components.td,{children:[`One exact checkpoint, as `,_jsx(_components.code,{children:`\u003crun>/\u003cstep>`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--step`})}),_jsx(_components.td,{children:`integer`}),_jsx(_components.td,{}),_jsx(_components.td,{children:`The checkpoint at one absolute step. Refuses when two runs saved a checkpoint at that step.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--gpu`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`the smallest type that fits the model`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`l4`}),`, `,_jsx(_components.code,{children:`l40s`}),`, `,_jsx(_components.code,{children:`a100-80gb`}),`, `,_jsx(_components.code,{children:`h100`}),`, or `,_jsx(_components.code,{children:`h200`}),`. With `,_jsx(_components.code,{children:`--remote`}),` only.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--gpus`})}),_jsx(_components.td,{children:`integer`}),_jsx(_components.td,{children:`the smallest count that fits the model`}),_jsxs(_components.td,{children:[`The number of GPUs, with `,_jsx(_components.code,{children:`--remote`}),` only. A count above 8 spans several nodes.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--remote`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Submit the eval to a GPU instance and return`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--push`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Run here and upload the results after each state change`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--smoke`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Score the first 3 tasks. A smoke run never counts as a result or a baseline.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--dry-run`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print the target, the eval, and the settings, and start nothing`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--yes`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsxs(_components.td,{children:[`Skip the `,_jsx(_components.code,{children:`launch?`}),` prompt`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`--model`}),`, `,_jsx(_components.code,{children:`--checkpoint`}),`, and `,_jsx(_components.code,{children:`--step`}),` exclude each other. `,_jsx(_components.code,{children:`--remote`}),` does not combine with `,_jsx(_components.code,{children:`--push`}),` or `,_jsx(_components.code,{children:`--dry-run`}),`.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` eval`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --checkpoint`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` train-07/200`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --remote`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`7`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`run support-triage/eval-03 (eval)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`eval support-escalation-dev@1 (dev, 200 tasks)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`target checkpoint train-07/200 on Qwen/Qwen3-8B @ a09a3545`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`gpus h100 x1`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`launch? [y/N] y`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Submitted support-triage/eval-03 to gpu-4f2a. Follow it with: monte run logs support-triage/eval-03 -f`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Error`}),_jsx(_components.th,{children:`Cause`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_REFUSAL`})}),_jsxs(_components.td,{children:[`No `,_jsx(_components.code,{children:`\u003ceval>`}),` and more than one development eval, no `,_jsx(_components.code,{children:`--model`}),` and no checkpoint, or `,_jsx(_components.code,{children:`--step`}),` matches two checkpoints`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_PREFLIGHT`})}),_jsx(_components.td,{children:`The eval changed its fingerprint, or the run lock is held`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_NOT_FOUND`})}),_jsx(_components.td,{children:`No such experiment, eval, or checkpoint`})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`monte-grade`,children:`monte grade`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`monte grade \u003cexperiment>/\u003crun> --into \u003cexperiment> [--eval \u003ceval>]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` [--remote | --push] [--dry-run] [--yes] [--json]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Score the replies of an eval run again with the grader of another experiment’s eval, and record a new eval run there. No model is served: the command reads the original run’s `,_jsx(_components.a,{href:`/observability/traces`,children:`traces`}),`. The new run counts as a result in the target experiment, and a regraded baseline is a baseline there. `,_jsx(_components.a,{href:`/environments/graders#fix-a-grader-and-regrade`,children:`Fix a Grader and Regrade`}),` shows the full sequence.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Flag`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Default`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`\u003cexperiment>/\u003crun>`})}),_jsx(_components.td,{children:`argument`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The original eval run`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--into`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`required`}),_jsx(_components.td,{children:`The target experiment. It cannot be the experiment of the original run.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--eval`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`the one eval whose tasks match`}),_jsx(_components.td,{children:`The eval in the target experiment, when more than one has the same tasks`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--remote`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Run the grader on a GPU instance and return, for a judge model that needs a GPU`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--push`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Run here and upload the results after each state change`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--dry-run`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print the original run, the target eval, and the task match, and start nothing`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--yes`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsxs(_components.td,{children:[`Skip the `,_jsx(_components.code,{children:`launch?`}),` prompt`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`--json`})}),_jsx(_components.td,{children:`flag`}),_jsx(_components.td,{children:`off`}),_jsx(_components.td,{children:`Print JSON on stdout`})]})]})]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`monte`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` grade`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage/eval-03`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --into`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` support-triage-v2`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`4`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`4`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`tasks 200 of 200 match by task hash settings match`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`launch? [y/N] y`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`support-triage-v2/eval-03 succeeded support-escalation-dev=0.57±0.07 (200 tasks, 0 served)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Error`}),_jsx(_components.th,{children:`Cause`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_NOT_COMPARABLE`})}),_jsx(_components.td,{children:`The target eval has other tasks, or other eval settings, than the original eval`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`E_REFUSAL`})}),_jsx(_components.td,{children:`The original run is not an eval run, is a smoke run, or has no stored traces`})]})]})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Evaluations","description":"Experiments, evals, baselines, and regrades","href":"/commands/evaluations"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"monte experiment create","slug":"monte-experiment-create","depth":2,"children":[]},{"title":"monte experiment list","slug":"monte-experiment-list","depth":2,"children":[]},{"title":"monte experiment show","slug":"monte-experiment-show","depth":2,"children":[]},{"title":"monte experiment baselines","slug":"monte-experiment-baselines","depth":2,"children":[]},{"title":"monte experiment delete","slug":"monte-experiment-delete","depth":2,"children":[]},{"title":"monte experiment sync","slug":"monte-experiment-sync","depth":2,"children":[]},{"title":"monte experiment push","slug":"monte-experiment-push","depth":2,"children":[]},{"title":"monte eval","slug":"monte-eval","depth":2,"children":[]},{"title":"monte grade","slug":"monte-grade","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":[],"subdomain":"monte-inc","siteName":"Monte Platform","title":"Evaluations - Monte Platform","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-10-30T21:06:09.281Z","originalFileLocation":"commands/evaluations.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Experiments, evals, baselines, and regrades\">\u003cmeta data-mint-head name=\"application-name\" content=\"Monte Platform\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Evaluations - Monte Platform\">\u003cmeta data-mint-head property=\"og:description\" content=\"Experiments, evals, baselines, and regrades\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Monte Platform\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.trymonte.ai/commands/evaluations\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://monte-inc.mintlify.app/_mintlify/api/og/v2?division=Commands\u0026amp;title=Evaluations\u0026amp;description=Experiments%2C+evals%2C+baselines%2C+and+regrades\u0026amp;theme=73df20490d3a6aec36f9f19b\u0026amp;signature=7c5c962203c3a1c095e02ad38b99217ecf6355886cb959bb3f2381dbd734b318\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Evaluations - Monte Platform\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Experiments, evals, baselines, and regrades\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://monte-inc.mintlify.app/_mintlify/api/og/v2?division=Commands\u0026amp;title=Evaluations\u0026amp;description=Experiments%2C+evals%2C+baselines%2C+and+regrades\u0026amp;theme=73df20490d3a6aec36f9f19b\u0026amp;signature=7c5c962203c3a1c095e02ad38b99217ecf6355886cb959bb3f2381dbd734b318\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/monte-inc/P2VsY79V6xu55Vfe/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Monte Platform\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#315594\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.trymonte.ai/commands/evaluations\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/commands/evaluations.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.trymonte.ai/#organization\",\"name\":\"Monte Platform\",\"url\":\"https://docs.trymonte.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/monte-inc/gDE0uEcoDx7_moLr/logo/light.png?fit=max\u0026auto=format\u0026n=gDE0uEcoDx7_moLr\u0026q=85\u0026s=fe282530810a529d346a3d34b18dfa1e\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.trymonte.ai#website\",\"name\":\"Monte Platform\",\"url\":\"https://docs.trymonte.ai\",\"publisher\":{\"@id\":\"https://docs.trymonte.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.trymonte.ai/commands/evaluations#webpage\",\"url\":\"https://docs.trymonte.ai/commands/evaluations\",\"name\":\"Evaluations\",\"description\":\"Experiments, evals, baselines, and regrades\",\"dateModified\":\"2026-10-30T21:06:09.281Z\",\"isPartOf\":{\"@id\":\"https://docs.trymonte.ai#website\"},\"breadcrumb\":{\"@id\":\"https://docs.trymonte.ai/commands/evaluations#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.trymonte.ai/commands/evaluations#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Commands\",\"item\":\"https://docs.trymonte.ai/commands/setup\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Evaluations\",\"item\":\"https://docs.trymonte.ai/commands/evaluations\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://docs.trymonte.ai/commands/evaluations#article\",\"headline\":\"Evaluations\",\"name\":\"Evaluations\",\"description\":\"Experiments, evals, baselines, and regrades\",\"url\":\"https://docs.trymonte.ai/commands/evaluations\",\"mainEntityOfPage\":{\"@id\":\"https://docs.trymonte.ai/commands/evaluations#webpage\"},\"dateModified\":\"2026-10-30T21:06:09.281Z\",\"publisher\":{\"@id\":\"https://docs.trymonte.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.trymonte.ai#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Monte Platform\",\"creator\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","siteName":"Monte Platform","nav":{"global":null,"tabs":[{"tab":"Platform","icon":"layer-group","pages":[{"title":"Platform","href":""}]},{"tab":"CLI","icon":"terminal","groups":[{"group":"Commands","pages":[{"title":"Setup","href":"/commands/setup"},{"title":"Training","description":"Training runs and checkpoints","href":"/commands/training"},{"title":"Evaluations","href":"/commands/evaluations"},{"title":"Environments","description":"Environments and their versions","href":"/commands/environments"},{"title":"Datasets","href":"/commands/datasets"},{"title":"Serving","href":"/commands/serving"},{"title":"Observability","href":"/commands/observability"}]},{"group":"MCP","pages":[{"title":"MCP Server","href":"/mcp/overview"}]}]}]},"basePath":"","title":"Evaluations - Monte Platform","buildId":"dpl_DeHjVRGfFnhYQwHnfC2pjeXXv8h9","deploymentId":"dpl_DeHjVRGfFnhYQwHnfC2pjeXXv8h9","dynamics":[],"chunks":[]}
