> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serving

> Serve a checkpoint behind an endpoint

A deployment serves one model behind an OpenAI-compatible endpoint: a checkpoint, a base model, or a LoRA adapter on a base deployment. With BYOC, it runs on GPU instances in the team's account. A deployment runs, and is billed, until it is deleted or scaled to zero.

## Choose a Checkpoint

Deploy only a checkpoint that beat the [baseline](https://docs.trymonte.ai/evaluation/experiments.md#take-the-baselines) on the held-out evals. `monte checkpoint show` prints those scores.

## Deploy a Fine-Tuned Model

```bash theme={null}
monte deployment create support-triage/train-07 --name support-triage \
  --cloud gcp --gpu h100 --min-replicas 1 --max-replicas 4
```

`support-triage/train-07` names the checkpoint that the run saved at its last step, `train-07/200`. `--name` is also the `model` value in each request. Without `--gpu`, Monte picks the smallest GPU type that fits the model and its context.

The command checks each servable file against its sha256: the safetensors weights, or the PEFT adapter of a LoRA run. If a file fails, the command refuses with `E_VERIFICATION`. Upload the weights again with `monte checkpoint push`, then run the command again.

The [training config](https://docs.trymonte.ai/training/configs.md) decides whether a checkpoint holds full weights or a LoRA adapter.

| | Full model | LoRA adapter |
| - | - | - |
| **What the checkpoint holds** | All weights of the model | Low-rank weights on top of a base model |
| **GPU use** | One server for each deployment | Many adapters share one base deployment |

Without `--base`, an adapter gets its own base-model server that other adapters cannot share. Callers need an [API key](https://docs.trymonte.ai/serving/endpoints.md#authenticate).

## Deploy a Base Model

`--model` deploys a Hugging Face model with no training. Each create takes exactly one of `<checkpoint>` or `--model`.

```bash theme={null}
monte deployment create --model Qwen/Qwen3-8B --name qwen3-8b \
  --max-adapters 8 --max-adapter-rank 64
```

## Add Adapters to a Base

`--base` deploys a LoRA checkpoint onto a running base deployment.

```bash theme={null}
monte deployment create support-triage/train-11 --name support-triage-dpo --base qwen3-8b
```

Each adapter is its own deployment with its own name, and requests for different adapters share the base model's GPUs. `monte key create --deployment support-triage-dpo` limits a key to one adapter.

* `--max-adapters` sets how many adapters stay in GPU memory at once (default 8). Other adapters load from CPU memory when a request needs them.
* `--max-adapter-rank` sets the highest adapter rank that the base accepts (default 64). An adapter's LoRA rank must not exceed it, and a higher maximum takes more GPU memory.
* An adapter deployment has no replicas of its own. Its requests count toward the base deployment's scaling, and scaling flags refuse with `E_REFUSAL` on an adapter.
* An adapter works only on the base model that it was trained on.

## Check Status

```bash theme={null}
monte deployment show support-triage
```

| Status | Description |
| - | - |
| `pending` | Monte waits for a GPU instance |
| `loading` | The server loads the weights |
| `ready` | The endpoint accepts requests |
| `updating` | A new version or GPU type starts. The current version serves requests. |
| `idle` | Zero replicas. The next request starts one. |
| `paused` | Zero replicas and a maximum of zero. Requests return `503`. |
| `failed` | The server did not start. `monte deployment show` prints the reason. |

`monte deployment list` prints every deployment. `monte deployment logs support-triage -f` follows the model server's logs.

## Run Outside Monte

`monte checkpoint download` writes the servable weights of a checkpoint in the Hugging Face layout. A LoRA run gives a PEFT adapter that vLLM can load.

```bash theme={null}
monte checkpoint download support-triage/train-07/200 --dest ./models/train-07
vllm serve Qwen/Qwen3-8B --enable-lora --max-lora-rank 64 \
  --lora-modules support-triage=./models/train-07
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.