Endpoints

Each deployment serves an OpenAI-compatible endpoint. Code that calls the OpenAI chat completions API needs only a new base URL, an API key, and the deployment name as model.

Find the Endpoint URL

monte deployment show prints the endpoint URL. All public deployments in one cloud share the URL, and model selects the deployment. With BYOC, the public URL resolves to a load balancer in the team’s account.

export MONTE_BASE_URL=$(monte deployment show support-triage --json | jq -r .endpoint)
export MONTE_API_KEY=<key that monte key create printed>

Choose a Public or Private Endpoint

	Public	Private
Address	A public URL with TLS	A private IP in the team’s network
Who can reach it	Any caller with an API key	Callers inside the team’s network, with an API key
Set with	The default	`monte deployment create --private`, with BYOC only

Authenticate

Each request sends an API key as a bearer token in the Authorization header. A key works for all deployments of the team, or only for the deployments that it names. A gateway next to the model servers checks each key against the key hashes that it caches. Requests never pass through the control plane, and they continue when the control plane is unreachable.

CLI
Console

monte key create --name support-triage-prod --deployment support-triage
monte key list
monte key revoke support-triage-prod

monte key create prints the key once. Without --deployment, the key works for all deployments, including later ones. Serving Commands lists the flags.

Send a Request

POST /v1/chat/completions takes the OpenAI chat completions format.

curl "$MONTE_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $MONTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "support-triage",
    "messages": [
      {"role": "system", "content": "Answer from the knowledge base, or escalate the ticket with one category."},
      {"role": "user", "content": "My files stopped syncing after the last update."}
    ],
    "max_completion_tokens": 400
  }'

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["MONTE_BASE_URL"] + "/v1",
    api_key=os.environ["MONTE_API_KEY"],
)

response = client.chat.completions.create(
    model="support-triage",
    messages=[
        {"role": "system", "content": "Answer from the knowledge base, or escalate the ticket with one category."},
        {"role": "user", "content": "My files stopped syncing after the last update."},
    ],
    max_completion_tokens=400,
)
print(response.choices[0].message.content)

Request Fields

Field	Type	Description
`model`	string	Required. The deployment name.
`messages`	array	Required. The conversation so far. Each message has a `role` (`system`, `user`, `assistant`, or `tool`) and a `content`.
`max_completion_tokens`	integer	The maximum number of tokens to generate. `max_tokens` is the older name, and it works too.
`temperature`	number	Sampling temperature, from 0 to 2
`top_p`	number	Nucleus sampling probability mass
`n`	integer	The number of choices to generate. The default is 1.
`stop`	string or array	One or more sequences where generation stops.
`seed`	integer	A seed for sampling. Repeatable output is not guaranteed.
`logprobs`	boolean	Return the log probability of each output token.
`response_format`	object	`{"type": "json_schema", "json_schema": {...}}` makes the reply match a JSON schema.
`stream`	boolean	Send the reply as server-sent events. The default is `false`.
`stream_options`	object	`{"include_usage": true}` adds a last chunk with the token counts.
`tools`	array	Function definitions that the model can call.
`tool_choice`	string or object	`none`, `auto`, `required`, or one named function.

An omitted sampling field takes the model’s own generation defaults. Tool calls work for model families with a tool-call chat template.

Response Fields

Field	Description
`id`	The ID of this completion
`model`	The deployment that served the request
`choices[].message`	The generated message. A tool call arrives in `tool_calls`.
`choices[].finish_reason`	`stop` at a natural end or a stop sequence, `length` at the token limit, `tool_calls` when the model calls a tool
`usage`	Token counts for the prompt, the completion, and the total

Each response also carries two headers: Monte-Version names the deployment version that served it, such as v2, and Monte-Request-Id identifies the request in logs.

Use an Eval Harness

Any harness that calls the OpenAI API can score a deployment. Set its base URL to the endpoint and its model to the deployment name. Monte does not record these scores; for a score compared with a baseline, package the tasks as an eval environment.

Select an Adapter

An adapter deployment has its own name, so a request selects it with model.

response = client.chat.completions.create(
    model="support-triage-dpo",
    messages=[{"role": "user", "content": "How many devices does the Pro plan allow?"}],
)

Stream the Response

With stream set to true, the endpoint sends server-sent events. Each event is a chat.completion.chunk whose choices[].delta holds the next part of the message. The stream ends with data: [DONE].

curl "$MONTE_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $MONTE_API_KEY" \
  -H "Content-Type: application/json" \
  -N \
  -d '{"model": "support-triage", "stream": true,
       "messages": [{"role": "user", "content": "How do I reset my password?"}]}'

stream = client.chat.completions.create(
    model="support-triage",
    messages=[{"role": "user", "content": "How do I reset my password?"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

List Models

GET /v1/models lists the deployments that the key can call. Each id is a value for model.

curl "$MONTE_BASE_URL/v1/models" -H "Authorization: Bearer $MONTE_API_KEY"

{"object": "list", "data": [{"id": "support-triage", "object": "model", "context_length": 32768}]}

Handle Errors

An error response has an HTTP status and an error object with a message, a type, and a string code.

{"error": {"message": "The prompt and max_completion_tokens exceed the context length of 32768", "type": "invalid_request_error", "code": "context_length_exceeded"}}

Status	Cause
`400`	A missing field, a bad value, or a prompt longer than the context
`401`	The API key is missing or revoked
`403`	The key does not name this deployment
`404`	No deployment has the name in `model`
`429`	The queue holds `--max-queue` requests (default 256). Retry after the `Retry-After` time.
`503`	The deployment has zero replicas, is starting, or is paused. Except when paused, the request starts a replica. Retry after the `Retry-After` time.
`504`	The request did not finish in 600 seconds, queue time included

Scaling and Versions explains the queue and cold starts.

Get Started

Training

Inference

Datasets

Environments

Evaluation

Observability

Security

Find the Endpoint URL

Choose a Public or Private Endpoint

Authenticate

Send a Request

Request Fields

Response Fields

Use an Eval Harness

Select an Adapter

Stream the Response

List Models

Handle Errors

​Find the Endpoint URL

​Choose a Public or Private Endpoint

​Authenticate

​Send a Request

​Request Fields

​Response Fields

​Use an Eval Harness

​Select an Adapter

​Stream the Response

​List Models

​Handle Errors

Find the Endpoint URL

Choose a Public or Private Endpoint

Authenticate

Send a Request

Request Fields

Response Fields

Use an Eval Harness

Select an Adapter

Stream the Response

List Models

Handle Errors