Sebastigo J. Forrnosinho and Maria da Graqa M. Miguel University of Coimbra 3000 Coimbra, Portugal
Markov Chains for Plotting the Course of Complex Reactions
Concentrations of molecular species during a chemical reaction are random functions of time. and, conseauentlv. chemical kinetics may be formulated 'in a probabiiistic & stochastic framework ( I ). Continuous and discrete time stochastic processes have extended the field of statistical thermodynamics to time dependent systems in order to predict rates from molecular properties and to obtain solutions for chemical systems with a small number of reacting molecules. However, such processes can also be applied to the search of deterministic solutions for complex reactions. Several methods for plotting quite rigorously the course of complex reactions are available in the literature, but very few (2) can be employed by the students with nothing more than a simple pocket calculator. The use of stochastic matrices and Markov chains provides a useful educational tool for the study of complex chemical reactions, including oscillatory systems, without either solving differential equations or supplying closed form rate equations. Miller (3) has applied discrete Markov chains to the study of first-order processes. Here we intend to extend such an approach to second and third-order processes and to establish Markov chains as a useful tool for the study of any kind of complex chemical reactions. The Markov Chain Method Let us consider the stochastic Drocess studied in classical probability theory, namely a sequence of independent random variables, XI, Xz, . . .X,, representing the results of n independent trials. Let El, Ez, . . .E,, represent the possible outcomes (states of the svstem) a t each trlal. Since the trials are of observing the sequence of independent, the outcomes EjI, E,s, . . . E,,, is simply the product of the probabilities p,(pj = P(Xj = El) associated with the outcomes PIE~I, Ej2.. . . Ej") = P ~ I P ~.PI" P..
Markov has eeneralized the classical situation bv assumine that the outcome of any trial depends only on thebutcome 07 the directly preceding trial. Hence, in this case, conditional probabilities, p,,, associated with every pair of outcomes have to he considered. The random variables of a Markov chain are clearly not independent but its dependence extends over the period between two trials, i.e., one unit of time. The concept of a Markov chain is obtained from an empirical process associated with svstems whase state changes with time according to some probability law, in such a manner that the probability of the system going from a given state ELat a time n to a state E, a t a time n 1depends only on the state Ei at time n, but is independent of the states of the system at times prior t o n . Therefore, Markov chains are convenient models to generate the growth or decay of populations, namely concentration curves for chemical reactions ( 4 ) . Let us consider a system of microstates al, az, . . .a, a t times 0,1, . . . n, . . . . If a t a time n the system is in a microstate aj,
+
582 1 Journal of Chemical Education
+
i t will then jump a t a time n 1to a possible new microstate according to a set of probabilities pi, which represent the probability of the outcome aj on any given experiment, given that the outcome a; occurred on the preceding experiment. The numbers pi, are also called transition probabilities and can he presented easily on a stochastic matrix, i.e., a square matrix with nonnegative entries such that the sum of the entries in each row is unity ( 4 ) a,
az am
a,
a2
PI^
~
L!,
p?
1
2
Pmn
.. .. .. .. . .. .. .. . . .. .. .. . .. .. .. ..
m
,=x, PU = 1
am
%"I
Pmm
In reaction kinetics the probabilities pi, represent the probability that a molecule ai will change into a molecule a,; the probability that ai will remain unchanged is pji. If the initial state of the ensemble is represented by the matrix (2) lla~lo[azlo[aslo... [a&l where [ail0 is the concentration of the molecular species ai at t = 0, the state after one transition (one unit of time) is the product of the two matrices of eqn. ( 2 ) and eqn. (1).The state after a second transition is obtained by multiplying the matrix state after the first transition by the transition matrix. By continuing such a sequence, one can obtain the probable state of the system after each transition and by relating each transition with a unit of time one can draw graphs for the concentrations of the different molecular species against time. For example the concentration of the molecule ai at an instant t 1 is related to the concentration of the molecular species at an instant t by
+
In chemical kinetics the transition probahi1itit.s can he rclated with the frartion of molecules with iutfirient enerrs lu react or with the rate constants for the chemical process&. If rate constants are used in the transition matrix, then the units of the rate constants will determine the duration of time (unit of time) covered by each transition. Probabilities and rate constants can be related for a first order process where [A] = [Aloeckli For a unit of time and with k l
