Evals engineer

A person or team whose job is to measure AI systems using verified test sets, benchmarks and regression evals.

Also called

AI product manager A person responsible for AI features built into products.
Product engineer A role which entails responsibility to design and build features through production to meet a user need.
Agent manager A person whose job is to oversee a group of agents at work, by setting their tasks and standards, checking the quality of their output, and fixing them when they fail.
Full-stack builder A person who uses AI tools to take a product from idea to launch alone, doing the work of a product manager, a designer and an engineer.