What is the difference between QA and evals for an AI product? QA asks whether the software does what the spec says. Evals ask whether the product tells the truth. Deterministic software fails loudly, with an exception and an alert. An LLM product fails politely: the answer arrives fluent, confident, formatted, and wrong, and nothing crashes.
Closing that gap takes 3 things. A scenario library of the questions users will actually ask, each with the shape of a correct answer and the data it must come from, which is the PRD of an AI product. Validation against live production data rather than fixtures, because real accounts are half-connected and contradictory in ways fixtures never are. And a gate rather than a report, meaning the suite runs on every prompt change and model swap, and a regression blocks the ship.
Every AI product works in the demo. The demo is a handpicked question, a warm model, and an audience that wants to believe. Production is a stranger asking about their own data with money on the line. The discipline that closes the gap between those 2 moments is evaluation, and most product teams still treat it as QA's odd cousin instead of what it actually is: the product decision.
How do AI products fail differently?
Deterministic software fails loudly. An exception, a stack trace, an alert. An LLM product fails politely: the answer arrives fluent, confident, formatted, and wrong. Nothing crashes and no alert fires. The user simply learns the product cannot be trusted with their own numbers, and that lesson, once taught, is nearly impossible to unteach. I wrote about this dynamic in The AI Trust Gap: the first sessions of an AI product are a trial, and a single confident wrong answer is most of the verdict.
The scenario library is the spec
Before the first prompt gets written, enumerate the questions users will actually ask. Not the questions the team hopes they will ask. Source them from support tickets, sales calls, onboarding sessions, and the search box. Write each one down with the shape of a correct answer and the data it must come from.
That library is the PRD of an AI product. The old spec described screens and states. This one describes questions and truths, and the PM owns it the same way. It is also the most precise piece of product writing most PMs will ever do, because vague scenarios produce unfalsifiable answers, and unfalsifiable answers cannot be graded.
Why validate against production instead of fixtures?
Clean test data lies. Fixtures are complete, consistent, and nothing like a real account, where integrations are half-connected, settings contradict each other, and history has holes. Run every scenario against live production data before release, account by account, and grade the answers against what is actually true for that account.
Then ship only the scenarios that pass. The ones that fail are not embarrassments to hide. They are the roadmap, ranked by how often real users would have hit them, and holding them back at launch is what lets everything you did ship be trusted.
A gate, not a report
An eval suite that produces a dashboard is a report, and reports get skimmed. The suite has to sit in the release path: every prompt change, model swap, and tool addition reruns the library, and a regression blocks the ship. LLM outputs are not deterministic, so a change that improves 3 answers can silently break 12 others, and without the gate nobody finds out until users do.
The suite also cannot freeze. A scenario set written at launch and never touched again turns into a benchmark of the past instead of a guard on the present. Every production failure becomes a new scenario, so the library stays a living map of how the product actually fails.
The stakes on that gate rise sharply once the product is priced per outcome, because a failed scenario stops being a bug and becomes an invoice nobody sends. I worked that through in When Resolved Becomes a Price.
What does the PM own?
The scenario portfolio and where it came from. The pass bar per scenario, because good enough on a billing question and good enough on a churn-risk summary are different standards. The decision to hold a failing scenario back, which is a scope call, not a technical one. R&D owns how the answers get produced. The eval suite is the contract between the 2 sides, and it is a better contract than any spec document, because it executes.
Missing evaluation infrastructure is also the most common reason a promising agent pilot never reaches production, which is the pattern across the published surveys in Production Is an Org Problem.
The question that decides retention
QA asks whether the software does what the spec says. Evals ask whether the product tells the truth. For AI products those are different questions, and the second one decides whether users come back. Treating evaluation as the product decision costs a slower week 1. What it buys is a user who stops double-checking the output, and no apology issued later has ever produced that.