Using Markov Chains to Model Genetic

Algorithms
Tarundeep Singh Dhot
Dept of ECE
Concordia University
Montreal, QC H3G 1M8

A Markov Chain is essentially a discrete-time stochastic process which can be

defined as a sequence of random values whose probabilities at a time interval depends on
the value of the number at the previous time. In order to implement a Markov Chain, a
Transition Matrix is defined which indicates the behaviour of the stochastic process
represented by the Markov Chain and if the previous state of the process was known, we
can come up with a distribution over the next state the process will enter (Chapman-
Kolmogorov Equations).

Markov Chain theory offers a sound framework to understand and explore the
potential and behaviour of GAs. Since GAs move from generation to generation i.e.
through successive populations with each population dependent on the previous one, we
can represent a GA as a Markov Chain by letting the current state of the GA to be the
current population, as the current population depends only on the previous generation
which is analogous to the definition of a Markov Chain. The formulation of the transition
matrix which dictates how the GA moves from one state to the next is based on the
genetic operators (selection, recombination and mutation) used by the GA. The action of
recombination and mutation operators is purely a function of the probability of crossover
and mutation while the action of selection operator encompasses information about the
fitness function.

It is important to note the role of elitism here as survivor selection mechanism is

elitist here, which means that the best individuals of a population (present generation) are
passed as it is to the next generation so as to ensure presence of the optimum in its
population (next generation). Thus, the value of the best individuals in each population is
a non decreasing sequence (bounded from above) ensuring optimum. This is evident from
the Elitist Theorem which proves that under some broad conditions, elitist GAs will
eventually have a population that contains the optimum.

To sum it up, we see that GAs can be modelled using Markov models where
occurrence of each individual in the population can be regarded as a state of a Markov
model although the number of states is enormously large. Even though such modelling is
limited to cases problems with limited size of their transition matrix, but it is still seen as
a significant effort in developing a general theory as well as research in the field of
modelling GAs.