Distil to a small model only after the frontier baseline is measured on the same eval

Written in May after x-slm-edge-classifier, which fine-tuned a 7B model for in-store product classification and compared it to a frontier baseline that had not been measured until the fourth week of the experiment. When the baseline was finally run, the fine-tuned model was 3 points behind it and 2 points behind a stock 32B open-weight model with a good prompt (c-slm-edge-tuning-1, c-slm-edge-tuning-2). The experiment had spent three weeks chasing a gap that did not exist.

The recommendation was procedural: measure first. It held, and it is still correct as far as it goes. It is superseded because r-open-weight-tier answers the question this one only orders: for classification and extraction, the stock open-weight tier is the default and a fine-tune is the exception, justified only by a measured gap on a held-out eval (c-slm-edge-tuning-4).

The edge case that keeps this alive as a footnote: genuinely offline inference on in-store hardware, where a 32B model does not fit. There, distillation to a 3–7B model is still the route, and this recommendation's rule applies — measure the target eval on the frontier and on the stock small model before touching a training run. g-fine-tune-domain-slm carries the fuller account.