AI FOR VET • MODULE 3

Model Complexity: Underfitting, Overfitting & L2 Regularization

How polynomial degree and Ridge penalty (λ) control generalization.

🎯 Learning Outcomes / Ishodi učenja:
  • Recognize underfitting (degree d = 1) on non-linear datasets.
  • Observe overfitting (high degree) where the curve memorizes noise.
  • Understand how L2 Ridge regularization (λ) prevents wild oscillations.
  • Monitor Mean Squared Error (MSE) on both training and validation sets.

A straight line cannot capture periodic or parabolic sensor signals (underfitting). However, a high-degree polynomial bends through every noise point (overfitting). Regularization (λ) penalizes extreme weights and restores smooth generalization.

J(w) = MSE + λ ∑ w_j²

The L2 regularization penalty balances fit quality with model simplicity.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Underfitting: Set degree d = 1. Compare training MSE vs validation MSE.
  2. Overfitting: Set degree d = 7 with λ = 0. Notice the wild spikes at boundaries.
  3. Regularization: Increase λ to 0.05. Observe how the curve smooths and validation MSE drops.