What is an eval in LLM development?

A repeatable test that scores your LLM system's output against known-good expectations — the unit test equivalent for model behavior. An eval suite runs a golden set of real inputs through your pipeline and checks outputs with deterministic rules (exact match, schema validation, rubric checks) or a validated LLM judge, so prompt and model changes can't regress quality silently.

When should I use LLM-as-judge instead of deterministic checks?

Only for qualities rules can't score — tone, helpfulness, whether an answer truly addresses the question. Use deterministic checks first for everything they can cover, then add a judge with bias mitigations: swap answer positions, anchor it to a written rubric, and validate a sample of its verdicts against human grades before trusting it.

How many evals do I need to start?

Twenty real examples with cheap assertions, running in CI, beat any elaborate platform you haven't built. Grow the set every time production surprises you — the suite should accumulate every input that has ever fooled the system.

Evals: The Unit Tests Your LLM App Doesn't Have

A one-word prompt edit once took a workflow from reliable to quietly broken for four days. No stack trace, no error rate spike, no alert — the model just started answering a slightly different question with total confidence. A user noticed before we did, which is the most embarrassing sentence an engineer can type.

That's the moment evals stop being an academic topic. Your prompt is code. It executes on every request. It has regressions, edge cases, and dependencies that shift under you. It just doesn't have tests — yet.

Golden sets come from traffic, not imagination

The foundation of every useful eval suite is a golden set: real inputs paired with what a correct output looks like. The key word is *real*. Sit with your logs and pull the questions users actually asked — including the weird ones, the typos, the ones phrased like a text message to a friend. Fifty real examples beat five hundred synthetic ones, because synthetic examples share your assumptions, and your assumptions are exactly what breaks.

Tip

Every time production surprises you, that input goes into the golden set the same day. Your eval suite should be a museum of everything that has ever fooled the system.

Cheap evals first — most failures are boring

Teams skip evals because they imagine building an LLM-judging-LLM platform with dashboards. Skip that. Most production failures are catchable with deterministic checks that cost nothing:

Exact match or contains — did the classifier output one of the allowed labels?
Schema validation — is the extraction valid JSON with the required fields, and are the dates actually dates?
Regex and bounds — is the amount a number, is the summary under the length cap, did it leak the system prompt?
Rubric checks — a short list of yes/no questions a script can answer: cites a source? refuses when it should?

evals.ts

// an eval is just a test where the assertion knows about language
for (const ex of goldenSet) {
  const out = await pipeline(ex.input);
  results.push({
    id: ex.id,
    parses: isValidTicket(out),               // schema: deterministic
    label: out.category === ex.expected.category, // exact: deterministic
    grounded: ex.mustCite.every((s) => out.body.includes(s)), // contains
  });
}
const pass = results.filter((r) => r.parses && r.label).length;
console.log(`${pass}/${results.length} — fail the build under 95%`);

LLM-as-judge: useful, biased, verify it

For fuzzy qualities — tone, helpfulness, whether an answer actually addresses the question — you'll eventually use a model to grade a model. It works, but the judge arrives with documented biases: it favors longer answers, it favors the first option shown, and it grades its own model family generously. Defang them: swap answer positions and average, anchor the judge to a written rubric instead of 'rate 1–10', and spot-check a sample of judge verdicts against a human until you trust the correlation.

An eval suite you didn't validate is a second untested LLM app grading your first one.

Wire it into CI or it didn't happen

The entire point is catching regressions *before* users do, which means evals run where tests run: on the pull request that changes the prompt, the model version, or the retrieval config. Set a threshold, fail the build below it, and keep a small always-on sample in production scoring live traffic so drift — a silent model update, a shifting input mix — shows up as a sloped line instead of a support ticket.

I've written before that prompting is just writing clearly, and that your agent needs a leash. Evals are the third leg: proof. Write twenty boring ones this week. They will embarrass you immediately, which is exactly the point.

Key takeaways

01Build golden sets from real logged traffic — synthetic examples share your assumptions, and assumptions are what break.
02Cheap deterministic checks (exact match, schema, rubrics) catch most failures before you need LLM-as-judge.
03Evals run in CI on every prompt or model change, with a threshold that fails the build. Otherwise they're decoration.

FAQ

AIEvalsTesting