> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymonte.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tasks

> Task rows and grader fields

A task is one row of a JSONL file: one JSON object on each line. Each attempt starts from one row.

## Task Row Format

| Field | Read by | Contents |
| - | - | - |
| `responses_create_params.input` | The model | The conversation: a system message and the customer's ticket |
| `responses_create_params.tools` | The model | The tools it can call, each with a JSON schema. The [server](https://docs.trymonte.ai/environments/servers.md) serves a tool of the same name. |
| Other top-level fields | The grader | The expected result. The model never sees them. |

The `support_escalation` grader reads four fields:

| Field | Type | Description |
| - | - | - |
| `should_escalate` | bool | The ticket must go to a person |
| `expected_category` | string | The category that an escalation must name: `money`, `legal`, `dataloss`, `cancel`, `angry`, or `kbgap`. Empty for a resolved ticket. |
| `expected_facts` | list of strings | The facts that a correct reply states. Empty for an escalation. |
| `kb_entry` | int | The knowledge base entry that answers the ticket, or `-1` |

## Example Row

This row must be resolved from the knowledge base with the fact `1 hour`. In the file, the row is on one line.

```json theme={null}
{
  "responses_create_params": {
    "input": [
      {"role": "system", "content": "Answer from the knowledge base with search_kb, or call escalate_to_human with one category."},
      {"role": "user", "content": "How long does the password reset link stay valid?"}
    ],
    "tools": [
      {"type": "function", "name": "search_kb", "description": "Search the knowledge base.",
       "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}},
      {"type": "function", "name": "escalate_to_human", "description": "Send the ticket to a person.",
       "parameters": {"type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"]}}
    ]
  },
  "should_escalate": false,
  "expected_category": "",
  "expected_facts": ["1 hour"],
  "kb_entry": 3
}
```

## Task Files

```text theme={null}
monte-gym/
├── resources_servers/support_escalation/
│   ├── configs/support_escalation.yaml   # declares the training rows
│   └── data/train.jsonl                  # training rows
└── benchmarks/
    ├── support_escalation/               # the held-out eval
    │   ├── config.yaml
    │   ├── prepare.py                    # writes the task file
    │   └── data/support_escalation_benchmark.jsonl
    └── support_escalation_dev/           # the development eval, same layout
```

The [server config](https://docs.trymonte.ai/environments/servers.md#server-config) declares training rows as a task file of `type: train`. An eval config declares one task file of `type: benchmark`, with the script that writes it:

```yaml theme={null}
config_paths:
  - resources_servers/support_escalation/configs/support_escalation.yaml

support_escalation_benchmark_simple_agent:
  _inherit_from: support_escalation_simple_agent
  responses_api_agents:
    simple_agent:
      datasets:
      - name: support_escalation
        type: benchmark
        jsonl_fpath: benchmarks/support_escalation/data/support_escalation_benchmark.jsonl
        prepare_script: benchmarks/support_escalation/prepare.py
        prompt_config: null
        num_repeats: 1
```

> [!WARNING]
>
> Never put eval tasks in a training file. A model trained on its eval tasks scores high on that eval and nowhere else.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.