# Human-as-judge

An eval where a person reviews a system's output and judges its quality.

This can be unavoidable when it comes to niche needs like style or preference. Human grades are the standard other judges are checked against, though people often disagree with each other.
Human grades are also the starting point to calibrate an LLM-as-judge.

Also called: Human evaluation, Human review, Human rating

Kind of: Eval

Often confused with: Human-in-the-loop

Source: https://redfinchadvisory.com/lexicon/human-as-judge (version 1.0, changed 2026-10-21)