AI FOR VET • MODULE 2

Evaluating Binary Classifiers: Confusion Matrix

Why accuracy is misleading on imbalanced datasets, and how threshold (θ) balances trade-offs.

🎯 Learning Outcomes / Ishodi učenja:
  • Calculate True Positives (TP), False Positives (FP), False Negatives (FN), True Negatives (TN).
  • Understand the trade-off between Precision and Recall.
  • Observe why 98% accuracy can still be useless in quality control.
  • Tune decision threshold (θ) for medical diagnosis vs spam filtering.

In factory inspection where only 2% of units are defective, a naive model that predicts "All OK" gets 98% accuracy while catching zero defects! The confusion matrix and F1-score reveal the real performance.

Precision = TP / (TP + FP)  |  Recall = TP / (TP + FN)  |  F1 = 2 · (P · R) / (P + R)

Balancing false alarms (FP) against missed defects (FN).

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Defect detection: Move threshold θ left. What happens to Recall and Precision?
  2. Medical screening: Switch scenario to Medical. Why is high Recall (≥ 0.95) paramount?
  3. Spam filter: Switch to Spam. Why must Precision be high (≥ 0.90) to avoid blocking real emails?