To build a Markov Chain you need two basic components:
- The state space : which you already have - {3.3*10^5, 1.4*10^6,2.4*10^6,3.5*10^6,4.6*10^6,5.7*10^6}. Since there are 6 states, your Transition Probability Matrix (TPM) will be 6×6.
- The transition mechanism : i.e. “if I am in state i now, what is the probability I move to state j next?”
Now to build TPM we need sequences that helps us calculate the probability of states. Let's try it with the help of an example:
- Suppose we have 3 dummy states just for simplicity: S={A,B,C}
- Collect a fake sequence of visits : A -> B -> B -> C -> A -> B -> C -> C
- Calculate number of transitions from each state:
From A : A -> A= 0 times, A -> B = 2 times, A -> C = 0 times : [0 2 0]
Probability = [0 2 0] / 2 = [0 1 0]
Similarly from states B and C we get:
From B : Probability = [0 1 2] / 3 = [0, 1/3, 2/3]
From C : Probability = [1 0 1] / 3 = [1/2, 0, 1/2]
The TPM matrix has rows indicating the current state and columns as next states. So the matrix will look like:
In this way, you can create Transition Probability Matrix (TPM) by calculating probabilities of current state on the basis of next states.
Hope this helps you, let me know if you have any more doubts.
