simulate

Consider this theoretical, right-stochastic transition matrix of a stochastic process.

Create the Markov chain that is characterized by the transition matrix P.

P = [ 0   0  1/2 1/4 1/4  0   0 ;
      0   0  1/3  0  2/3  0   0 ;
      0   0   0   0   0  1/3 2/3;
      0   0   0   0   0  1/2 1/2;
      0   0   0   0   0  3/4 1/4;
     1/2 1/2  0   0   0   0   0 ;
     1/4 3/4  0   0   0   0   0 ];
mc = dtmc(P);

Plot a directed graph of the Markov chain. Indicate the probability of transition by using edge colors.

figure;
graphplot(mc,'ColorEdges',true);

Simulate a 20-step random walk that starts from a random state.

rng(1); % For reproducibility
numSteps = 20;
X = simulate(mc,numSteps)

X = 21×1

     3
     7
     1
     3
     6
     1
     3
     7
     2
     5
     6
     2
     3
     7
     1
      ⋮

X is a 21-by-1 matrix. Rows correspond to steps in the random walk. Because X(1) is 3, the random walk begins at state 3.

Visualize the random walk.

figure;
simplot(mc,X);

Create a four-state Markov chain from a randomly generated transition matrix containing eight infeasible transitions.

rng('default'); % For reproducibility 
mc = mcmix(4,'Zeros',8);

mc is a dtmc object.

Plot a digraph of the Markov chain.

figure;
graphplot(mc);

State 4 is an absorbing state.

Run three 10-step simulations for each state.

x0 = 3*ones(1,mc.NumStates);
numSteps = 10;
X = simulate(mc,numSteps,'X0',x0);

X is an 11-by-12 matrix. Rows corresponds to steps in the random walk. Columns 1–3 are the simulations that start at state 1; column 4–6 are the simulations that start at state 2; columns 7–9 are the simulations that start at state 3; and columns 10–12 are the simulations that start at state 4.

For each time, plot the proportions states that are visited over all simulations.

figure;
simplot(mc,X)