Serving

These commands serve a model behind an OpenAI-compatible endpoint and control who can call it. Serving explains the sequence.

Command	Description
`monte deployment create`	Deploy a checkpoint, a base model, or an adapter
`monte deployment list`	List deployments
`monte deployment show`	Show one deployment, its endpoint, and its metrics
`monte deployment update`	Change scaling, or move traffic to a new version
`monte deployment logs`	Print or follow the model server’s logs
`monte deployment rollback`	Return to an earlier version
`monte deployment delete`	Remove a deployment
`monte key create`	Create an inference API key
`monte key list`	List keys
`monte key revoke`	Revoke a key

monte deployment create

monte deployment create (<checkpoint> | --model <hf-id>) --name <name> [--base <deployment>]
  [--cloud <name>] [--gpu <type>] [--gpus-per-replica <n>] [--min-replicas <n>] [--max-replicas <n>]
  [--max-queue <n>] [--max-adapters <n>] [--max-adapter-rank <n>] [--request-logs on|off] [--private] [--json]

Deploy a checkpoint or a base model as version v1 of a new deployment. Give exactly one of <checkpoint> or --model. --base loads a LoRA adapter into a running base deployment. Before it deploys a checkpoint, the command checks each servable file against its sha256: the safetensors weights, or the PEFT adapter of a LoRA run.

Flag	Type	Default	Description
`<checkpoint>`	argument	none	`<experiment>/<run>/<step>`, or `<experiment>/<run>` for the last checkpoint of the run
`--model`	string	none	A Hugging Face model ID, deployed as a base deployment with no training
`--name`	string	required	The deployment name. Requests send it as `model`.
`--base`	string	none	A running base deployment to load the adapter into
`--cloud`	string	the only connected cloud	The cloud that runs the deployment
`--gpu`	string	the smallest GPU type that fits the model and its context	`l4`, `l40s`, `a100-80gb`, `h100`, or `h200`
`--gpus-per-replica`	integer	`1`	GPUs for each replica: 1, 2, 4, or 8
`--min-replicas`	integer	`1`	The lowest replica count. `0` allows scale to zero.
`--max-replicas`	integer	`1`	The highest replica count
`--max-queue`	integer	`256`	Requests that wait for a replica before the endpoint returns `429`
`--max-adapters`	integer	`8`	With `--model` only. Adapters in GPU memory at once.
`--max-adapter-rank`	integer	`64`	With `--model` only. The highest LoRA rank that the base deployment accepts.
`--request-logs`	string	`on`	Keep request logs in artifact storage for 30 days, or `off` to keep none
`--private`	flag	off	Serve on a private IP in the team’s network. BYOC only.
`--json`	flag	off	Print JSON on stdout

monte deployment create support-triage/train-07 --name support-triage --cloud gcp --gpu h100 --min-replicas 1 --max-replicas 4

deployment  support-triage v1
checkpoint  support-triage/train-07/200 (LoRA adapter on Qwen/Qwen3-8B)
cloud       gcp (us-central1)  gpu h100 x1  replicas 1-4
status      pending

Error	Cause
`E_REFUSAL`	Both or neither of `<checkpoint>` and `--model`, the name is taken, `--base` serves another base model, the adapter rank is above the base’s `--max-adapter-rank`, or `--private` without BYOC
`E_VERIFICATION`	A servable file fails its sha256 check. Upload the weights again with `monte checkpoint push`.
`E_NOT_FOUND`	No such checkpoint, model, base deployment, or cloud

monte deployment list

monte deployment list [--cloud <name>] [--json]

List deployments with their status, version, checkpoint, replicas, and endpoint URL.

Flag	Type	Default	Description
`--cloud`	string	all clouds	Only the deployments in one cloud
`--json`	flag	off	Print JSON on stdout

monte deployment list

NAME                  STATUS   VERSION   CHECKPOINT                    BASE      REPLICAS   ENDPOINT
support-triage        ready    v2        support-triage/train-11/300   -         2 (1-4)    https://inference.us-central1.helpdesk.trymonte.ai
qwen3-8b              ready    v1        Qwen/Qwen3-8B                 -         1 (1-2)    https://inference.us-central1.helpdesk.trymonte.ai
support-triage-dpo    ready    v1        support-triage/train-11/300   qwen3-8b  on base    https://inference.us-central1.helpdesk.trymonte.ai

monte deployment show

monte deployment show <name> [--metrics] [--json]

Show one deployment: status, endpoint URL, versions and traffic shares, replicas, and queue.

Flag	Type	Default	Description
`<name>`	argument	required	The deployment
`--metrics`	flag	off	Add the last hour’s requests, error rates, latency, and tokens per second for each version
`--json`	flag	off	Print JSON on stdout. The URL is in `endpoint`.

monte deployment show support-triage --metrics

deployment  support-triage  ready
endpoint    https://inference.us-central1.helpdesk.trymonte.ai
model       support-triage  context 32768
versions    v2  support-triage/train-11/300  10%
            v1  support-triage/train-07/200  90%
replicas    2 (min 1, max 4, target concurrency 32)  gpu h100 x1
queue       0 waiting (max 256)

VERSION  REQUESTS  4XX    5XX    TTFT P50  LATENCY P50  LATENCY P95  TOKENS/S
v2       412       0.2%   0.0%   180 ms    1.9 s        3.4 s        2210
v1       3690      0.3%   0.1%   175 ms    1.8 s        3.3 s        2190

Status	Description
`pending`	Monte waits for a GPU instance
`loading`	The server loads the weights
`ready`	The endpoint accepts requests
`updating`	A new version or GPU type starts. The current version serves requests.
`idle`	Zero replicas. The next request starts one.
`paused`	Zero replicas and a maximum of zero. Requests return `503`.
`failed`	The server did not start. The output prints the reason.

monte deployment update

monte deployment update <name> [--min-replicas <n>] [--max-replicas <n>] [--target-concurrency <n>]
  [--schedule <window>]... [--scale-down-delay <time>] [--max-queue <n>] [--gpu <type>]
  [--gpus-per-replica <n>] [--checkpoint <checkpoint>] [--traffic <percent>] [--request-logs on|off] [--json]

Change how a deployment scales, or start a new version and move traffic to it. Monte moves the share when the new version is ready. Scaling and Versions explains the rules.

Flag	Type	Default	Description
`<name>`	argument	required	The deployment
`--min-replicas`	integer	unchanged (`1` at create)	The lowest replica count. `0` allows scale to zero. `0` with `--max-replicas 0` pauses the deployment.
`--max-replicas`	integer	unchanged (`1` at create)	The highest replica count
`--target-concurrency`	integer	unchanged (`32` for a new deployment)	Concurrent requests for each replica. Above it, Monte adds a replica.
`--schedule`	string	unchanged (none for a new deployment)	A window with its own minimum, such as `"Mon-Fri 07:00-19:00 America/New_York min=2"`. Repeat it to add windows.
`--scale-down-delay`	duration	unchanged (`10m` for a new deployment)	The time below the target before Monte removes a replica, and the idle time before the last one stops
`--max-queue`	integer	unchanged (`256` at create)	Requests that wait for a replica before the endpoint returns `429`
`--gpu`	string	unchanged	A new GPU type. New replicas start before the old ones stop.
`--gpus-per-replica`	integer	unchanged	GPUs for each replica: 1, 2, 4, or 8
`--checkpoint`	string	none	A checkpoint for a new version, as `<experiment>/<run>[/<step>]`
`--traffic`	integer	`100` with `--checkpoint`, else unchanged	The share of requests, in percent, for the newest version
`--request-logs`	string	unchanged (`on` at create)	Keep request logs in artifact storage for 30 days, or `off` to keep none
`--json`	flag	off	Print JSON on stdout

monte deployment update support-triage --checkpoint support-triage/train-11 --traffic 10

deployment  support-triage  updating
version     v2  support-triage/train-11/300  loading -> ready in 1m48s
traffic     v2 10%, v1 90%

Error	Cause
`E_REFUSAL`	The checkpoint has another base model than an adapter deployment’s base, or a scaling flag names an adapter deployment
`E_VERIFICATION`	A servable file of the checkpoint fails its sha256 check

monte deployment logs

monte deployment logs <name> [--version <version>] [--tail <n>] [-f] [--json]

Print the model server logs of a deployment, with the replica and the version on each line. The logs never include prompts or completions.

Flag	Type	Default	Description
`<name>`	argument	required	The deployment
`--version`	string	all versions	Only the logs of one version, such as `v2`
`--tail`	integer	all	Print only the last `n` lines first
`-f`	flag	off	Keep the output open and print new lines
`--json`	flag	off	Print JSON lines on stdout

monte deployment logs support-triage --version v2 --tail 2

2026-10-28 08:14:02 UTC  v2  replica-1  Loaded weights in 41.2 s
2026-10-28 08:14:05 UTC  v2  replica-1  Server ready on port 8000

monte deployment rollback

monte deployment rollback <name> [--to <version>] [--json]

Send all traffic to the previous version at once, or to the version that --to names. A version with no replicas returns 503 until a replica starts.

Flag	Type	Default	Description
`<name>`	argument	required	The deployment
`--to`	string	the previous version	A version, such as `v1`
`--json`	flag	off	Print JSON on stdout

monte deployment rollback support-triage

traffic     v1 100% (support-triage/train-07/200), v2 0%

monte deployment delete

monte deployment delete <name> [--yes] [--json]

Stop all replicas, remove every version, and free the name. Requests to the name then return 404. The checkpoints and the runs stay.

Flag	Type	Default	Description
`<name>`	argument	required	The deployment
`--yes`	flag	off	Skip the prompt
`--json`	flag	off	Print JSON on stdout

monte deployment delete support-triage --yes

Deleted support-triage: 2 versions, 2 replicas stopped.

Error	Cause
`E_IN_USE`	Adapter deployments run on this base deployment. Delete them first.

monte key create

monte key create --name <name> [--deployment <name>]... [--json]

Create an API key for the inference endpoint. The command prints the key once, and Monte keeps only its hash.

Flag	Type	Default	Description
`--name`	string	required	A name for the key, unique in the team
`--deployment`	string	all deployments, including later ones	Limit the key to one deployment. Repeat it for more. Other deployments return `403`.
`--json`	flag	off	Print JSON on stdout

monte key create --name support-triage-prod --deployment support-triage

key         support-triage-prod  (deployments: support-triage)
secret      mk_live_4Jt9Qe0vX2pLr8sWb6YcN1aH
This is the only time the key is shown.

monte key list

monte key list [--json]

List the team’s keys with name, prefix, deployments, and last use. The command never prints a full key.

Flag	Type	Default	Description
`--json`	flag	off	Print JSON on stdout

monte key list

NAME                  PREFIX         DEPLOYMENTS      CREATED      LAST USED
support-triage-prod   mk_live_4Jt9   support-triage   2026-10-21   2026-10-28 08:11 UTC
staging               mk_live_Zq01   all              2026-10-02   2026-10-19 17:40 UTC

monte key revoke

monte key revoke <name> [--yes] [--json]

Revoke a key. Requests with the key get 401 within one minute.

Flag	Type	Default	Description
`<name>`	argument	required	The key name
`--yes`	flag	off	Skip the prompt
`--json`	flag	off	Print JSON on stdout

monte key revoke staging --yes

Revoked staging (mk_live_Zq01). Last used 2026-10-19 17:40 UTC.

Commands

MCP

monte deployment create

monte deployment list

monte deployment show

monte deployment update

monte deployment logs

monte deployment rollback

monte deployment delete

monte key create

monte key list

monte key revoke

​monte deployment create

​monte deployment list

​monte deployment show

​monte deployment update

​monte deployment logs

​monte deployment rollback

​monte deployment delete

​monte key create

​monte key list

​monte key revoke

monte deployment create

monte deployment list

monte deployment show

monte deployment update

monte deployment logs

monte deployment rollback

monte deployment delete

monte key create

monte key list

monte key revoke