AI FOR VET • MODULE 9

Responsible AI: Dataset Representation & Hidden Bias

Why 95% overall test accuracy can conceal discriminatory failure on minority groups.

🎯 Learning Outcomes / Ishodi učenja:
  • Understand training set vs independent test set evaluation.
  • Examine the danger of under-represented subgroups (e.g. Factory B at 5%).
  • Demonstrate why high average accuracy can hide biased performance.
  • Apply EU AI Act requirements: representative data and group-disaggregated metrics.

If a quality-control model is trained on 95% parts from Factory A and only 5% from Factory B, an aggregate score of 95% can look great while completely failing for Factory B! Data quality and representation matter far more than sheer volume.

Overall Accuracy = w_A · Acc_A + w_B · Acc_B  |  Minority Disparity = |Acc_A − Acc_B|

Evaluating performance separately for every demographic/operational subgroup is mandatory for trustworthy AI.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Hidden minority failure: Keep Factory B at 5%. Overall accuracy looks high (~95%), but check Factory B accuracy!
  2. Balance representation: Increase Factory B share to 35%. How does Factory B accuracy jump?
  3. Memorization trap: Set k = 1. Training accuracy is 100%, but generalization on test data is unstable.