# Eval

A measurement of how well an AI system does something (tone, helpfulness, accuracy, safety), usually as a score, graded by code, a model or a person.

An eval doesn't have to be structured or a comparison: a one-off tone check on a single customer-service transcript is an eval. 
Not limited to models; you can eval a harness, a context setup, or other AI infrastructure. Comparison and benchmarking are uses of evals, not the definition. 
Edge case: an eval can have a deterministic answer (did the model solve the math problem?) and still be an eval, because it measures capability.

Also called: Evaluations, Model evaluation, AI testing

Avoid: Vibe check, Vibes-based eval

Related: Red teaming, Observability, Reward hacking

Source: https://redfinchadvisory.com/lexicon/eval (version 1.0, changed 2026-10-21)