An AI team is looking for a way to compare several internal candidate models under fair conditions. What is the name of the practice of comparing and evaluating multiple models or methods under the same conditions using a common standard dataset and metrics?

1 / 1
Select an answer
CorrectA

Explanation

Question Overview

A question about choosing the practice of comparing models under common conditions.

Requirements to satisfy
  • 1「under the same conditions using a common standard dataset and metrics」A side-by-side comparison with conditions kept equal = benchmark
Per-option explanation
ACorrect

Benchmark

Correct. A benchmark is the practice of comparing and evaluating multiple models or methods side by side under the same conditions using a common standard dataset and metrics. It enables a fair comparison.

BIncorrect

A/B testing

A/B testing is a method that splits production users across two variants and compares the results.

It is a comparison in a real environment, not a side-by-side evaluation using a standard dataset, so this is incorrect.

CIncorrect

Human evaluation

Human evaluation is a method in which humans judge the quality of output.

It is effective for qualitative assessment, but it is not the practice of quantitative side-by-side comparison using a common dataset and metrics, so this is incorrect.

DIncorrect

Red teaming

Red teaming is an evaluation that elicits risks from an attacker's perspective.

It is a safety inspection, not a side-by-side comparison of performance, so this is incorrect.

Key Takeaway

Note the correct answer, benchmark.
- The practice of comparing and evaluating multiple models or methods side by side under the same conditions using a common standard dataset and metrics.
- By keeping conditions equal, it can fairly judge which is superior.

Related Links