AI FOR VET • MODULE 4

Comparing Instance-Based Learning with Axis-Parallel Trees

How k-NN and Decision Trees partition non-linear feature spaces.

🎯 Learning Outcomes / Ishodi učenja:
  • Observe organic curved boundaries in k-Nearest Neighbors.
  • Understand rectangular, axis-aligned partition boxes in Decision Trees.
  • Examine the effect of neighborhood size k (k=1 memorization vs large k oversmoothing).
  • Control tree complexity via maximum depth.

k-NN classifies each point by majority vote of its k closest neighbors, creating smooth boundaries. Decision Trees make sequential if/then threshold decisions, carving the space into rectangular boxes.

k-NN: majority vote among k closest  |  Decision Tree: split on feature x_i ≤ threshold

Different inductive biases produce very different decision regions.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Moons dataset with k-NN: Test k = 1 vs k = 5 vs k = 15. When is the boundary smoothest?
  2. Switch to Decision Tree: Increase depth from 1 to 5. Watch how tree partitions the moons.
  3. Circles dataset: Compare how easily k-NN vs Decision Trees handle concentric rings.