AI FOR VET • MODULE 1

Model Optimization: Gradient Descent

How machine learning models learn optimal weights and minimize error.

🎯 Learning Outcomes / Ishodi učenja:
  • Understand cost function J(w) as the measure of model error.
  • Observe how the gradient determines direction and magnitude of weight updates.
  • Investigate learning rate (α): underfitting vs convergence vs divergence.
  • Explore local vs global minima in non-convex optimization landscapes.

The core objective of supervised learning is finding parameters (weights $w$) that minimize the cost function $J(w)$. Think of navigating a foggy mountain into a valley: you check the ground slope beneath your feet and step in the direction of steepest descent.

w ← w − α · (dJ / dw)

Where $w$ is current weight, α is learning rate, and dJ/dw is the gradient slope.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Normal convergence: With profile Convex and α = 0.15, click Run. How many epochs to reach minimum at w = 2.0?
  2. Sluggish rate: Set α = 0.01. Observe how slow convergence becomes.
  3. Divergence: Increase α ≥ 0.85. Notice how the point overshoots and diverges into infinity.
  4. Local minimum trap: Select Non-convex and start from w = −2.2. Does it reach the local or global minimum?