A company is defining the fairness review procedure for a credit scoring model. The company wants to examine whether there are differences in accuracy or error rates across demographic groups such as age group, gender, and region. What is the name of this type of analysis?
Identify the analysis that examines performance differences across demographic groups.
k-fold cross-validation
This is incorrect. Cross-validation splits the data into multiple folds and alternates one fold for validation and the rest for training to estimate generalization performance. While the validation fold rotates, this technique is designed to measure overall model performance stably — it does not examine performance differences across demographic groups such as age or gender.
Outlier analysis
This is incorrect. Outlier analysis is used to identify data points that are far from the majority. It detects individual anomalous data points and does not examine performance differences across demographic groups.
Subgroup analysis
This is correct. Subgroup analysis separates data into demographic groups and examines whether there are differences in accuracy or error rates across those groups. It can detect unfair treatment of specific groups.
Correlation analysis
This is incorrect. Correlation analysis is a statistical technique that examines the strength of the relationship between variables. It examines relationships between variables, not differences in accuracy or error rates across demographic groups.
Subgroup analysis separates data by demographic groups such as age, gender, and region and examines whether there are differences in accuracy or error rates. It can detect unfairness where a specific group has lower performance even when overall metrics look good — an important tool for fairness validation.
· k-fold cross-validation: estimates generalization performance
· Outlier analysis: detects anomalous data points
· Correlation analysis: measures variable relationships
Each serves a different purpose.