Improvement

Training a model to where you want it is rarely a single run. You pick a method and some settings, train, look at what came back, and change something. Which methods work is not settled in advance either. It depends on the model you started from, the tasks you care about, and the compute you can spend. With that many variables in play, the record of what you changed matters as much as the changes themselves. Each one should be visible: what you altered, what it was compared against, and what came back. The figure below is a high-level view of that work with an agent driving it. Two things move at different speeds inside it: the model’s weights, and the method used to change them.

​

Moving the weights

Training updates the model’s weights using one Recipe. It is ordinary training: one of the methods described in Training, run against the train Split for a fixed number of steps. What comes out is a Checkpoint, which is the model’s saved state after those steps. The Recipe does not change while this runs. Every choice about the method is settled before the first step and holds until the Checkpoint is saved. This is the part usually called the inner loop.

​

Moving the method

The Checkpoint is then scored on the dev Split, and the result is written to the Ledger. The Ledger holds the Baseline and every Run since, so it is where you find out what a given Recipe did and how that compared to what came before. The agent reads that record and decides the next move, which is the part usually called the outer loop. A move is two choices made together: which Checkpoint to start from, and which Recipe to run on it. Those two choices are the arrow back into training, and running it produces one more Checkpoint. Nothing is replaced, so the record only ever grows. If the last Recipe produced a measurable improvement, continuing with it is the obvious move. If it did not, the agent changes something and tries again. That is a simplification. Deciding whether a Recipe actually helped takes more than one number: the interval around the score, gradient norms and the other training diagnostics, and whether the model has begun overfitting. A single improvement on its own says very little. This page describes the shape of the process rather than the analysis inside it. Every one of those decisions is made by reading dev. That is what makes dev useful, and it is also what disqualifies it as the number you report. A score you have optimized against many times stops being an independent measure of the model and starts reflecting the search that produced it. The test Split is held back for the final claim.

​

Continue, branch, or stage

Every move is one of three kinds. Continue. Run the same Recipe again on the newest Checkpoint. Branch. Start from a Checkpoint that is not the newest one. If a learning rate turned out too high, you return to the last Checkpoint that looked healthy and go a different way, and the arm that failed stays on the record. Stage. Change the kind of method rather than its settings. Supervised fine-tuning first, so the model becomes competent at the format and the tool-calling conventions, then reinforcement learning on top of those weights to sharpen it. All three appear below, in a deliberately simple picture of what one of these records could look like. Each box is a Checkpoint labelled with the step it was saved at, and each arrow is one training Run labelled with the Recipe that ran. The first arrow is a fine-tuning stage, the second is the boundary where the method changes to reinforcement learning, and the fork at 50 steps is a branch retried at a lower learning rate. All three are the same operation underneath: pick a Checkpoint, pick a Recipe. Because a Checkpoint can have more than one child, what builds up is a tree, and one pass through the figure at the top of this page is a single path down it.

​

Making the claim

The search runs until the model reaches the result you were after, or until the budget you set aside runs out. Monte does not make that call. Nothing exits the search automatically, and no rule decides that you have run enough. The choice of which Checkpoint to claim is not yours, though. The tool selects the one that scored highest on dev. That keeps the last decision consistent with every earlier one, all of which were also made by reading dev. That Checkpoint is then scored on test, and the result is the claim. It is the first read of test since the Baseline, and the last one Monte will allow. The rule is two per Cohort: a third is refused, and so is a claim at any step the tool did not select. Dev selects, test confirms. If the claim shows an improvement, the Checkpoint can be promoted. Promoting exports its servable folder as a checksum-verified copy, behind a confirmation you have to type out in full. If the claim shows no improvement, nothing is promoted and the Ledger keeps the record exactly as it stands. The tree still shows every method that was tried and what each one scored, so the work has an answer either way.

​

Related pages