Rate limits as product behavior, not just infra
Why LLM rate limits are product decisions about who waits, who fails, and who gets a degraded answer, and how to design them that way.
Blog
-
-
Golden sets that rot after the first model upgrade
Why LLM golden sets go stale on model upgrades, how to tell product regressions from scorer drift, and how to re-anchor the suite as part of the upgrade.
-
Cost attribution for LLM calls when finance starts asking
How to attach dollars to products and features so a month-end inference spike is a query, not a scavenger hunt.
-
Prompt versioning that survives more than one engineer
Treat prompts as named, immutable artifacts with rollout, rollback, and ownership so production behavior is explainable.
-
Why auth migrations and LLM rollouts look the same to operators
A guest post from the Click2Login team on the operating disciplines that auth migrations and LLM rollouts share, and what teams running one can borrow from the other.
-
What we mean by LLM evaluation in production
How to set up evaluation for an LLM feature so the team can ship changes with confidence, including what to test, what to ignore, and what the operational cadence looks like.
-
When your LLM features need ops, not more prompts
How to recognize the moment a team's LLM work has outgrown prompt engineering and needs the operational layer that turns experimental features into reliable production.