> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serving

> Deployments and inference API keys

These commands serve a model behind an OpenAI-compatible endpoint and control who can call it. [Serving](https://docs.trymonte.ai/serving/deploy.md) explains the sequence.

| Command | Description |
| - | - |
| [`monte deployment create`](#monte-deployment-create) | Deploy a checkpoint, a base model, or an adapter |
| [`monte deployment list`](#monte-deployment-list) | List deployments |
| [`monte deployment show`](#monte-deployment-show) | Show one deployment, its endpoint, and its metrics |
| [`monte deployment update`](#monte-deployment-update) | Change scaling, or move traffic to a new version |
| [`monte deployment logs`](#monte-deployment-logs) | Print or follow the model server's logs |
| [`monte deployment rollback`](#monte-deployment-rollback) | Return to an earlier version |
| [`monte deployment delete`](#monte-deployment-delete) | Remove a deployment |
| [`monte key create`](#monte-key-create) | Create an inference API key |
| [`monte key list`](#monte-key-list) | List keys |
| [`monte key revoke`](#monte-key-revoke) | Revoke a key |

## monte deployment create

```text theme={null}
monte deployment create (<checkpoint> | --model <hf-id>) --name <name> [--base <deployment>]
  [--cloud <name>] [--gpu <type>] [--gpus-per-replica <n>] [--min-replicas <n>] [--max-replicas <n>]
  [--max-queue <n>] [--max-adapters <n>] [--max-adapter-rank <n>] [--request-logs on|off] [--private] [--json]
```

Deploy a checkpoint or a base model as version `v1` of a new deployment. Give exactly one of `<checkpoint>` or `--model`. `--base` loads a LoRA adapter into a running base deployment. Before it deploys a checkpoint, the command checks each servable file against its sha256: the safetensors weights, or the PEFT adapter of a LoRA run.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<checkpoint>` | argument | none | `<experiment>/<run>/<step>`, or `<experiment>/<run>` for the last checkpoint of the run |
| `--model` | string | none | A Hugging Face model ID, deployed as a base deployment with no training |
| `--name` | string | required | The deployment name. Requests send it as `model`. |
| `--base` | string | none | A running base deployment to load the adapter into |
| `--cloud` | string | the only connected cloud | The cloud that runs the deployment |
| `--gpu` | string | the smallest GPU type that fits the model and its context | `l4`, `l40s`, `a100-80gb`, `h100`, or `h200` |
| `--gpus-per-replica` | integer | `1` | GPUs for each replica: 1, 2, 4, or 8 |
| `--min-replicas` | integer | `1` | The lowest replica count. `0` allows scale to zero. |
| `--max-replicas` | integer | `1` | The highest replica count |
| `--max-queue` | integer | `256` | Requests that wait for a replica before the endpoint returns `429` |
| `--max-adapters` | integer | `8` | With `--model` only. Adapters in GPU memory at once. |
| `--max-adapter-rank` | integer | `64` | With `--model` only. The highest LoRA rank that the base deployment accepts. |
| `--request-logs` | string | `on` | Keep request logs in artifact storage for 30 days, or `off` to keep none |
| `--private` | flag | off | Serve on a private IP in the team's network. BYOC only. |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte deployment create support-triage/train-07 --name support-triage --cloud gcp --gpu h100 --min-replicas 1 --max-replicas 4
```

```text theme={null}
deployment  support-triage v1
checkpoint  support-triage/train-07/200 (LoRA adapter on Qwen/Qwen3-8B)
cloud       gcp (us-central1)  gpu h100 x1  replicas 1-4
status      pending
```

| Error | Cause |
| - | - |
| `E_REFUSAL` | Both or neither of `<checkpoint>` and `--model`, the name is taken, `--base` serves another base model, the adapter rank is above the base's `--max-adapter-rank`, or `--private` without BYOC |
| `E_VERIFICATION` | A servable file fails its sha256 check. Upload the weights again with `monte checkpoint push`. |
| `E_NOT_FOUND` | No such checkpoint, model, base deployment, or cloud |

## monte deployment list

```text theme={null}
monte deployment list [--cloud <name>] [--json]
```

List deployments with their status, version, checkpoint, replicas, and endpoint URL.

| Flag | Type | Default | Description |
| - | - | - | - |
| `--cloud` | string | all clouds | Only the deployments in one cloud |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte deployment list
```

```text theme={null}
NAME                  STATUS   VERSION   CHECKPOINT                    BASE      REPLICAS   ENDPOINT
support-triage        ready    v2        support-triage/train-11/300   -         2 (1-4)    https://inference.us-central1.helpdesk.trymonte.ai
qwen3-8b              ready    v1        Qwen/Qwen3-8B                 -         1 (1-2)    https://inference.us-central1.helpdesk.trymonte.ai
support-triage-dpo    ready    v1        support-triage/train-11/300   qwen3-8b  on base    https://inference.us-central1.helpdesk.trymonte.ai
```

## monte deployment show

```text theme={null}
monte deployment show <name> [--metrics] [--json]
```

Show one deployment: status, endpoint URL, versions and traffic shares, replicas, and queue.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The deployment |
| `--metrics` | flag | off | Add the last hour's requests, error rates, latency, and tokens per second for each version |
| `--json` | flag | off | Print JSON on stdout. The URL is in `endpoint`. |

```bash theme={null}
monte deployment show support-triage --metrics
```

```text theme={null}
deployment  support-triage  ready
endpoint    https://inference.us-central1.helpdesk.trymonte.ai
model       support-triage  context 32768
versions    v2  support-triage/train-11/300  10%
            v1  support-triage/train-07/200  90%
replicas    2 (min 1, max 4, target concurrency 32)  gpu h100 x1
queue       0 waiting (max 256)

VERSION  REQUESTS  4XX    5XX    TTFT P50  LATENCY P50  LATENCY P95  TOKENS/S
v2       412       0.2%   0.0%   180 ms    1.9 s        3.4 s        2210
v1       3690      0.3%   0.1%   175 ms    1.8 s        3.3 s        2190
```

| Status | Description |
| - | - |
| `pending` | Monte waits for a GPU instance |
| `loading` | The server loads the weights |
| `ready` | The endpoint accepts requests |
| `updating` | A new version or GPU type starts. The current version serves requests. |
| `idle` | Zero replicas. The next request starts one. |
| `paused` | Zero replicas and a maximum of zero. Requests return `503`. |
| `failed` | The server did not start. The output prints the reason. |

## monte deployment update

```text theme={null}
monte deployment update <name> [--min-replicas <n>] [--max-replicas <n>] [--target-concurrency <n>]
  [--schedule <window>]... [--scale-down-delay <time>] [--max-queue <n>] [--gpu <type>]
  [--gpus-per-replica <n>] [--checkpoint <checkpoint>] [--traffic <percent>] [--request-logs on|off] [--json]
```

Change how a deployment scales, or start a new version and move traffic to it. Monte moves the share when the new version is `ready`. [Scaling and Versions](https://docs.trymonte.ai/serving/scaling-versions.md) explains the rules.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The deployment |
| `--min-replicas` | integer | unchanged (`1` at create) | The lowest replica count. `0` allows scale to zero. `0` with `--max-replicas 0` pauses the deployment. |
| `--max-replicas` | integer | unchanged (`1` at create) | The highest replica count |
| `--target-concurrency` | integer | unchanged (`32` for a new deployment) | Concurrent requests for each replica. Above it, Monte adds a replica. |
| `--schedule` | string | unchanged (none for a new deployment) | A window with its own minimum, such as `"Mon-Fri 07:00-19:00 America/New_York min=2"`. Repeat it to add windows. |
| `--scale-down-delay` | duration | unchanged (`10m` for a new deployment) | The time below the target before Monte removes a replica, and the idle time before the last one stops |
| `--max-queue` | integer | unchanged (`256` at create) | Requests that wait for a replica before the endpoint returns `429` |
| `--gpu` | string | unchanged | A new GPU type. New replicas start before the old ones stop. |
| `--gpus-per-replica` | integer | unchanged | GPUs for each replica: 1, 2, 4, or 8 |
| `--checkpoint` | string | none | A checkpoint for a new version, as `<experiment>/<run>[/<step>]` |
| `--traffic` | integer | `100` with `--checkpoint`, else unchanged | The share of requests, in percent, for the newest version |
| `--request-logs` | string | unchanged (`on` at create) | Keep request logs in artifact storage for 30 days, or `off` to keep none |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte deployment update support-triage --checkpoint support-triage/train-11 --traffic 10
```

```text theme={null}
deployment  support-triage  updating
version     v2  support-triage/train-11/300  loading -> ready in 1m48s
traffic     v2 10%, v1 90%
```

| Error | Cause |
| - | - |
| `E_REFUSAL` | The checkpoint has another base model than an adapter deployment's base, or a scaling flag names an adapter deployment |
| `E_VERIFICATION` | A servable file of the checkpoint fails its sha256 check |

## monte deployment logs

```text theme={null}
monte deployment logs <name> [--version <version>] [--tail <n>] [-f] [--json]
```

Print the model server logs of a deployment, with the replica and the version on each line. The logs never include prompts or completions.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The deployment |
| `--version` | string | all versions | Only the logs of one version, such as `v2` |
| `--tail` | integer | all | Print only the last `n` lines first |
| `-f` | flag | off | Keep the output open and print new lines |
| `--json` | flag | off | Print JSON lines on stdout |

```bash theme={null}
monte deployment logs support-triage --version v2 --tail 2
```

```text theme={null}
2026-10-28 08:14:02 UTC  v2  replica-1  Loaded weights in 41.2 s
2026-10-28 08:14:05 UTC  v2  replica-1  Server ready on port 8000
```

## monte deployment rollback

```text theme={null}
monte deployment rollback <name> [--to <version>] [--json]
```

Send all traffic to the previous version at once, or to the version that `--to` names. A version with no replicas returns `503` until a replica starts.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The deployment |
| `--to` | string | the previous version | A version, such as `v1` |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte deployment rollback support-triage
```

```text theme={null}
traffic     v1 100% (support-triage/train-07/200), v2 0%
```

## monte deployment delete

```text theme={null}
monte deployment delete <name> [--yes] [--json]
```

Stop all replicas, remove every version, and free the name. Requests to the name then return `404`. The checkpoints and the runs stay.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The deployment |
| `--yes` | flag | off | Skip the prompt |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte deployment delete support-triage --yes
```

```text theme={null}
Deleted support-triage: 2 versions, 2 replicas stopped.
```

| Error | Cause |
| - | - |
| `E_IN_USE` | Adapter deployments run on this base deployment. Delete them first. |

## monte key create

```text theme={null}
monte key create --name <name> [--deployment <name>]... [--json]
```

Create an API key for the inference endpoint. The command prints the key once, and Monte keeps only its hash.

| Flag | Type | Default | Description |
| - | - | - | - |
| `--name` | string | required | A name for the key, unique in the team |
| `--deployment` | string | all deployments, including later ones | Limit the key to one deployment. Repeat it for more. Other deployments return `403`. |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte key create --name support-triage-prod --deployment support-triage
```

```text theme={null}
key         support-triage-prod  (deployments: support-triage)
secret      mk_live_4Jt9Qe0vX2pLr8sWb6YcN1aH
This is the only time the key is shown.
```

## monte key list

```text theme={null}
monte key list [--json]
```

List the team's keys with name, prefix, deployments, and last use. The command never prints a full key.

| Flag | Type | Default | Description |
| - | - | - | - |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte key list
```

```text theme={null}
NAME                  PREFIX         DEPLOYMENTS      CREATED      LAST USED
support-triage-prod   mk_live_4Jt9   support-triage   2026-10-21   2026-10-28 08:11 UTC
staging               mk_live_Zq01   all              2026-10-02   2026-10-19 17:40 UTC
```

## monte key revoke

```text theme={null}
monte key revoke <name> [--yes] [--json]
```

Revoke a key. Requests with the key get `401` within one minute.

| Flag | Type | Default | Description |
| - | - | - | - |
| `<name>` | argument | required | The key name |
| `--yes` | flag | off | Skip the prompt |
| `--json` | flag | off | Print JSON on stdout |

```bash theme={null}
monte key revoke staging --yes
```

```text theme={null}
Revoked staging (mk_live_Zq01). Last used 2026-10-19 17:40 UTC.
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.