Verified test set
A set of test cases that have had correct answers verified by a subject-matter expert.
When used for evals: cases should reflect your company's real work; synthetic cases are fine when real data can't be used (common in healthcare, e.g. synthetic patient or customer records), as long as they are representative. The expert sign-off is non-negotiable: the expert owns the answer key and ideally stays involved in reviewing how evals score against it.
Also called
- Golden Dataset
Kind of
- Eval A measurement of how well an AI system does something (tone, helpfulness, accuracy, safety), usually as a score, graded by code, a model or a person.
Often confused with
- Benchmark A fixed, standardized eval suite held constant so comparisons are fair: across models, across approaches, or over time.
Related
- Regression eval An eval designed to protect a working AI feature against unintended change.
How it connects
- Verified test set used by Eval