A/B test
An experiment where live users are split between two versions of a product or service and the results are compared.
Also called
- A/B testing
- Split test
- Online experiment
Kind of
- Eval A measurement of how well an AI system does something (tone, helpfulness, accuracy, safety), usually as a score, graded by code, a model or a person.
Often confused with
- Side-by-side eval An eval that compares the outputs of two or more models or systems next to each other to judge which is better.
Related
- Pilot A release of a product to a limited group of real users doing real work, for a limited time, with a planned short-term review of its viability or effectiveness.