AI FOR VET • MODULE 6

From Single Perceptrons to Multi-Layer Neural Networks

Overcoming linear separability limits to solve non-linear problems like XOR.

🎯 Learning Outcomes / Ishodi učenja:
  • Understand why a single perceptron cannot solve the XOR problem.
  • Observe how hidden layers bend decision boundaries into complex contours.
  • Compare activation functions (tanh, ReLU, sigmoid).
  • Watch backpropagation adjust connection weights in real-time.

A single neuron is a straight ruler: it can only draw one straight decision boundary. A hidden layer is a team of rulers: the output neuron combines their linear boundaries into curved, non-linear shapes capable of isolating complex clusters.

y = σ( ∑ w_i · x_i + b )  |  Hidden: h_j = f( ∑ w_{ij} x_i + b_j )

Activation functions introduce non-linearity, allowing deep networks to approximate any function.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. XOR with 0 hidden layers: Set layers = 0 and train. Notice accuracy stalls around 50%.
  2. Add 1 hidden layer: Set 1 hidden layer with 4 neurons. Train again: accuracy reaches 98%+!
  3. Circle in a circle: Switch dataset to Circles. Train with 4 neurons to enclose the inner cluster.