# Benchmark

A fixed, standardized eval suite held constant so comparisons are fair: across models, across approaches, or over time.

Benchmarks are most commonly thought of as public test sets kept by research groups and AI labs. Your company can and should have its own private benchmarks, built around the tasks you need done, and run evals against them when something new is released or updated. Private benchmarks also avoid a known problem with public ones: the test questions may have leaked into a model's training data.

Kind of: Eval

Often confused with: Verified test set

Related: Jagged frontier

Source: https://redfinchadvisory.com/lexicon/benchmark (version 1.0, changed 2026-10-21)