A golden set is supposed to be the stable reference for whether an LLM feature is still doing the job. Inputs, expected outputs, maybe a few scoring notes. Teams treat it like a unit test suite for language. Then the first model upgrade lands, half the cases flip, and nobody can say whether the product got worse or the reference went stale.
That failure mode is common enough that it deserves a name and a practice. Golden sets rot. They rot on a schedule that looks like model release cadence, not on a schedule that looks like product cadence.
In traditional software, a golden file is a byte-for-byte expected artifact. If the output changes, the test fails, and the engineer either fixes the code or updates the golden file with intent. The assumption underneath is that there is one correct answer, or at least one correct serialization of the answer.
LLM features break that assumption immediately. The "correct" answer is often a range of acceptable answers. Two responses can both be right and still share almost no tokens. Exact-match scoring against a stored string will fail a good upgrade and pass a bad one, depending on how the model happens to phrase things that day.
So most teams loosen the check. They store an exemplar answer and score with semantic similarity, an LLM-as-judge rubric, or a handful of assertions ("must mention the refund window", "must not invent a policy id"). That is the right direction. It is also where rot starts, because the exemplar and the rubric were calibrated against the old model's habits.
A golden set is not only the inputs. It is the inputs, the labels, the scorer, the thresholds, and the unspoken expectation of how the current model tends to answer. Change the model and you have changed an input to the evaluation system that the golden set never recorded.
How rot shows up after an upgrade
The first symptom is a noisy regression report. You swap models, rerun the suite, and thirty percent of cases move. Some scores go up. Some go down. The aggregate looks "about the same" if you squint. Product wants a ship decision by Friday. Engineering cannot explain which deltas are real regressions and which are the scorer reacting to a different writing style.
The second symptom is threshold theatre. Someone raises the pass bar because the new model is "better," or lowers it because the suite is "too strict for this model family." Either way, the golden set stops being a fixed gate and becomes a dial that absorbs disagreement.
The third symptom is silent drift in the labels themselves. An engineer updates a few expected answers so the suite turns green. That is sometimes correct. It is often just aligning the reference to the new model's preferred phrasing without checking whether users still get the outcome they need. The set looks maintained. The measurement has moved underneath the product.
None of this requires malice or laziness. It is what happens when a reference artifact assumes a stable generator and the generator is the thing you keep replacing.
Separate product truth from model style
The useful split is between outcomes the product owns and surface form the model owns.
Product truth is whether the answer cites the right policy, whether the extracted fields are present and typed correctly, whether the tool call sequence is legal, whether the refusal fires on the disallowed class of request. Those checks should be written as assertions that do not care about prose. Structured outputs help here. So do deterministic validators on JSON schemas, citation ids, and allowlisted tool names.
Model style is tone, length, hedging language, and the particular way a model explains a correct answer. Style will jump on every upgrade. If your golden set's pass criteria are mostly style-shaped, every upgrade will look like a quality event even when the product outcomes are unchanged.
A practical rule: if a human grader would accept both the old and new answers for the same ticket, the automated scorer should not fail the case solely because the wording moved. Put the energy into checks a human would also treat as decisive.
That does not mean style is irrelevant. Customer-facing products have voice requirements. Treat those as a separate track with explicit examples and a rubric that is reviewed when brand guidance changes, not as a silent passenger inside every functional golden case.
Exact string match is almost never the right primary metric for free text. Prefer checks that pin down the decision the feature is supposed to make. For extraction and classification, score fields, not paragraphs. For grounded Q&A, score citation presence and faithfulness checks against the retrieved context, not similarity to a stored essay. For agents, score whether the trajectory reached an allowed terminal state with the right side effects. Use LLM-as-judge only where the judgment criteria are written down and stable, and keep a small human-audited subset so you can tell when the judge itself drifted after a model change.
Thresholds should be set per case class, not as one global number that people argue about in Slack. A policy refusal case and a casual greeting case do not deserve the same bar. When an upgrade moves scores, look at the class-level deltas first. Global averages hide the failures that matter.
Rebuild the reference on a known cadence
A golden set that only gets attention when CI goes red will always lag the model. Treat re-anchoring as part of the upgrade procedure, the same way you treat canaries and rollback criteria.
Before cutting traffic, run the existing suite against the candidate model and quarantine every case that flipped. Split the quarantine into three buckets: clear product regression, clear scorer or label mismatch, and ambiguous. Clear regressions block the upgrade or force a prompt and tooling fix. Clear mismatches get label or assertion updates with a short note of why. Ambiguous cases get a human pass before anyone moves a threshold.
After the upgrade ships, sample live traffic under the new model and refresh a slice of the golden set from that sample. The point is not to grow the set forever. The point is to keep the reference distribution close to what production actually looks like under the model you are running now. A set frozen on last quarter's model and last quarter's traffic is a museum exhibit.
Version the golden set the way you version prompts. Record which model family and prompt revision the labels were last reconciled against. When those change, the set's "last reconciled" metadata should change too. If that field is older than the model in production, the suite is advisory at best.
A maintained golden set has an owner, a reconciliation date, a link to the model and prompt it was last aligned with, and a short changelog when labels or assertions move. Failures from production incidents get added as cases with the product outcome encoded as a check, not as a pasted "good" answer from whatever model was handy that afternoon.
It also has a known failure mode for upgrades: expect churn, quarantine it, classify it, then ship or fix. Teams that skip the quarantine step end up either freezing on an old model because the suite screams, or shipping with a green suite that no longer measures the product.
Golden sets are still worth having. They are one of the few ways to make LLM changes reviewable. They just are not durable the way classical golden files are. Plan for them to rot after the first model upgrade, and build the re-anchoring work into the upgrade itself. Otherwise the suite becomes a confidence ritual that fails the moment you need it most.