> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Endpoints

> Call a deployment through an OpenAI-compatible API

Each deployment serves an OpenAI-compatible endpoint. Code that calls the OpenAI chat completions API needs only a new base URL, an API key, and the deployment name as `model`.

## Find the Endpoint URL

`monte deployment show` prints the endpoint URL. All public deployments in one cloud share the URL, and `model` selects the deployment. With BYOC, the public URL resolves to a load balancer in the team's account.

```bash theme={null}
export MONTE_BASE_URL=$(monte deployment show support-triage --json | jq -r .endpoint)
export MONTE_API_KEY=<key that monte key create printed>
```

## Choose a Public or Private Endpoint

| | Public | Private |
| - | - | - |
| **Address** | A public URL with TLS | A private IP in the team's network |
| **Who can reach it** | Any caller with an API key | Callers inside the team's network, with an API key |
| **Set with** | The default | `monte deployment create --private`, with [BYOC](https://docs.trymonte.ai/deployment.md) only |

## Authenticate

Each request sends an API key as a bearer token in the `Authorization` header. A key works for all deployments of the team, or only for the deployments that it names. A gateway next to the model servers checks each key against the key hashes that it caches. Requests never pass through the control plane, and they continue when the control plane is unreachable.

> [tab] CLI
>
> ```bash theme={null}
> monte key create --name support-triage-prod --deployment support-triage
> monte key list
> monte key revoke support-triage-prod
> ```
>
> `monte key create` prints the key once. Without `--deployment`, the key works for all deployments, including later ones. [Serving Commands](https://docs.trymonte.ai/commands/serving.md#monte-key-create) lists the flags.

> [tab] Console
>
> 1. Click **Credentials** in the sidebar, under **Team**.
> 2. Click **Create Key**.
> 3. Type a name.
> 4. Select **All deployments**, or select the deployments that the key can call. **All deployments** includes deployments created later.
> 5. Copy the key. The Console shows it only once.

## Send a Request

`POST /v1/chat/completions` takes the OpenAI chat completions format.

```bash curl theme={null}
curl "$MONTE_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $MONTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "support-triage",
    "messages": [
      {"role": "system", "content": "Answer from the knowledge base, or escalate the ticket with one category."},
      {"role": "user", "content": "My files stopped syncing after the last update."}
    ],
    "max_completion_tokens": 400
  }'
```

```python Python theme={null}
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["MONTE_BASE_URL"] + "/v1",
    api_key=os.environ["MONTE_API_KEY"],
)

response = client.chat.completions.create(
    model="support-triage",
    messages=[
        {"role": "system", "content": "Answer from the knowledge base, or escalate the ticket with one category."},
        {"role": "user", "content": "My files stopped syncing after the last update."},
    ],
    max_completion_tokens=400,
)
print(response.choices[0].message.content)
```

### Request Fields

| Field | Type | Description |
| - | - | - |
| `model` | string | Required. The deployment name. |
| `messages` | array | Required. The conversation so far. Each message has a `role` (`system`, `user`, `assistant`, or `tool`) and a `content`. |
| `max_completion_tokens` | integer | The maximum number of tokens to generate. `max_tokens` is the older name, and it works too. |
| `temperature` | number | Sampling temperature, from 0 to 2 |
| `top_p` | number | Nucleus sampling probability mass |
| `n` | integer | The number of choices to generate. The default is 1. |
| `stop` | string or array | One or more sequences where generation stops. |
| `seed` | integer | A seed for sampling. Repeatable output is not guaranteed. |
| `logprobs` | boolean | Return the log probability of each output token. |
| `response_format` | object | `{"type": "json_schema", "json_schema": {...}}` makes the reply match a JSON schema. |
| `stream` | boolean | Send the reply as server-sent events. The default is `false`. |
| `stream_options` | object | `{"include_usage": true}` adds a last chunk with the token counts. |
| `tools` | array | Function definitions that the model can call. |
| `tool_choice` | string or object | `none`, `auto`, `required`, or one named function. |

An omitted sampling field takes the model's own generation defaults. Tool calls work for model families with a tool-call chat template.

### Response Fields

| Field | Description |
| - | - |
| `id` | The ID of this completion |
| `model` | The deployment that served the request |
| `choices[].message` | The generated message. A tool call arrives in `tool_calls`. |
| `choices[].finish_reason` | `stop` at a natural end or a stop sequence, `length` at the token limit, `tool_calls` when the model calls a tool |
| `usage` | Token counts for the prompt, the completion, and the total |

Each response also carries two headers: `Monte-Version` names the deployment version that served it, such as `v2`, and `Monte-Request-Id` identifies the request in logs.

## Use an Eval Harness

Any harness that calls the OpenAI API can score a deployment. Set its base URL to the endpoint and its model to the deployment name. Monte does not record these scores; for a score compared with a baseline, package the tasks as an [eval environment](https://docs.trymonte.ai/environments/overview.md#create-and-add-an-environment).

## Select an Adapter

An adapter deployment has its own name, so a request selects it with `model`.

```python theme={null}
response = client.chat.completions.create(
    model="support-triage-dpo",
    messages=[{"role": "user", "content": "How many devices does the Pro plan allow?"}],
)
```

## Stream the Response

With `stream` set to `true`, the endpoint sends server-sent events. Each event is a `chat.completion.chunk` whose `choices[].delta` holds the next part of the message. The stream ends with `data: [DONE]`.

```bash curl theme={null}
curl "$MONTE_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $MONTE_API_KEY" \
  -H "Content-Type: application/json" \
  -N \
  -d '{"model": "support-triage", "stream": true,
       "messages": [{"role": "user", "content": "How do I reset my password?"}]}'
```

```python Python theme={null}
stream = client.chat.completions.create(
    model="support-triage",
    messages=[{"role": "user", "content": "How do I reset my password?"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
```

## List Models

`GET /v1/models` lists the deployments that the key can call. Each `id` is a value for `model`.

```bash theme={null}
curl "$MONTE_BASE_URL/v1/models" -H "Authorization: Bearer $MONTE_API_KEY"
```

```json theme={null}
{"object": "list", "data": [{"id": "support-triage", "object": "model", "context_length": 32768}]}
```

## Handle Errors

An error response has an HTTP status and an `error` object with a `message`, a `type`, and a string `code`.

```json theme={null}
{"error": {"message": "The prompt and max_completion_tokens exceed the context length of 32768", "type": "invalid_request_error", "code": "context_length_exceeded"}}
```

| Status | Cause |
| - | - |
| `400` | A missing field, a bad value, or a prompt longer than the context |
| `401` | The API key is missing or revoked |
| `403` | The key does not name this deployment |
| `404` | No deployment has the name in `model` |
| `429` | The queue holds `--max-queue` requests (default 256). Retry after the `Retry-After` time. |
| `503` | The deployment has zero replicas, is starting, or is paused. Except when paused, the request starts a replica. Retry after the `Retry-After` time. |
| `504` | The request did not finish in 600 seconds, queue time included |

[Scaling and Versions](https://docs.trymonte.ai/serving/scaling-versions.md#queueing-and-cold-starts) explains the queue and cold starts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.