Reinforcement learning is an unsupervised learning model based on a reward function that rewards desired behaviors and punishes negative ones. The incentives from rewards hopefully makes progress on the desired outcome, which is all done automatically. The model works by iterating over “state”, in which a decision must be made.

Rewards are typically applied at “terminal state”, which is a desired final condition, usually marking the end of a learning outcome. No rewards are given after terminal state is reached. In the meantime, states in between the desired states have relatively low (or zero) rewards, to incentivize the algorithm to continue moving. Combining these low rewards with the time steps, the algorithm has a way to determine if it is in a suboptimal path (or endless loop).

Each step is a function of the state $s$, action $a$, reward $R,$ and next state $s'$,

$$ algorithm\rightarrow (s, a, R(s), s') $$

This approach is formally known as the Markov Decision Process (MDP). The future only depends on current state and not how the algorithm got there.

Defining the Model

Defining the Model Algorithm

Random (Stochastic) Environments

The State-value Function

Defining the Learning Algorithm

Refining the Learning Algorithm