The greedy rule
In each state s, the greedy policy selects the action with the largest Q-value: pi(s) = argmax over a of Q(s, a). The policy is read off the table, so it improves whenever the table improves.
Reading a policy from a small table
Consider a corridor with two states, A and B, and two actions, left and right. Suppose the learned table is Q(A, left) = 2.0, Q(A, right) = 5.5, Q(B, left) = 4.0, Q(B, right) = 1.5. Applying argmax in each state gives pi(A) = right and pi(B) = left. No extra training is needed to produce this policy; it is a direct read of the table.
Why a wrong early estimate gets fixed
Suppose Q(A, right) starts at 0 because nothing has been tried. The first time the agent takes right from A, it receives a reward and lands in a next state whose best value is already nonzero. That reward plus the discounted next-state value becomes the target, and the error is large and positive, so Q(A, right) jumps upward. Each later visit adds another correction. The estimate does not need to be right at the start; it only needs to be updated often enough that the errors shrink.
A boundary on the greedy rule
Greedy action selection is how we extract a decision from a finished table. It is not by itself a good way to act while the table is still being learned, because always taking the current best action can leave better alternatives untried. That problem is the subject of the next chapter.