Understanding Sink State Formation in Finite State Machines using the Prisoner's Dilemma

This paper investigates what behavior causes the formation of sink states in Finite State Machines (FSM) through the lens of the Iterated Prisoner's Dilemma (IPD) game. In the IPD, two players play multiple rounds where each round they can choose to cooperate or defect. A FSM playing the IPD will decide to cooperate or defect based on its trained arrangement of states and transitions. If all of the transitions from a state return back to that same state, that state is a sink state. This paper finds that the chance of training a sink state is relative to the method of training the FSM. Evolutionary learning techniques train sink states more often than reinforcement learning techniques in most scenarios. This investigation provides insight into the nature of finite state machines that could be applied in other domains.