A company is deciding on a format to quantitatively report the results of a fairness review of a credit model to executives and auditors. What is the collective name for the metrics used to numerically measure whether a model's predictions are fair across groups?
Choosing the collective name for metrics that measure fairness numerically.
Fairness metrics
Correct. Fairness metrics are the collective name for metrics that numerically measure whether predictions are fair across groups (for example, the difference in accuracy between groups). They are used for quantitative evaluation of bias.
Accuracy metrics
Accuracy metrics (such as accuracy and F1 score) measure how often the model's predictions are correct.
They are about overall performance, not the collective name for metrics that measure fairness across groups, so it is incorrect.
Regression metrics
Regression metrics (such as RMSE) measure the error of numeric predictions.
They are about prediction error, not metrics of fairness across groups, so it is incorrect.
Drift metrics
Drift metrics measure how far the distribution of input data has shifted from training time.
They are about change during operation, not metrics of fairness across groups, so it is incorrect.
Remember the correct answer, 'fairness metrics.'
・The collective name for metrics that numerically measure whether predictions are fair across groups (for example, the difference in accuracy or positive rate between groups).
・They allow bias to be evaluated quantitatively and can be computed with SageMaker Clarify and similar tools.