AI FOR VET • MODULE 8

Learning by Interaction: Q-Learning in Gridworlds

How an autonomous agent learns optimal policies from trial, reward and penalty.

🎯 Learning Outcomes / Ishodi učenja:
  • Understand States, Actions, Rewards, and the Q-table.
  • Observe exploration (ε) vs exploitation balance.
  • Investigate discount factor (γ): short-term gratification vs long-term goal.
  • Watch the agent learn the optimal shortest path through a maze.

Unlike supervised learning with given answers, an RL agent learns like training a puppy: it takes actions, receives rewards (+10 for goal, −0.1 per step, −10 for falling), and gradually learns which moves yield maximum cumulative reward.

Q(s, a) ← Q(s, a) + α [ r + γ max Q(s', a') − Q(s, a) ]

The Bellman equation updates expected future reward for each state-action pair.

🔬 Interactive Laboratory / Interaktivna laboratorija:

🔍 Guided Investigation / Istraživački zadatak:
  1. Maze exploration: Click Run and watch arrows appear in cells as the Q-table learns the path.
  2. Cliff walking: Switch map to Cliff. Set ε = 0.4 vs ε = 0.05. When does it fall less?
  3. Treasure dilemma: Switch map to Treasure. Set γ = 0.5 (takes small nearby reward) vs γ = 0.95 (takes big distant goal).