You can start with creating a square maxtrix of zeros. See this for more details:
M= zeros(n); %where n is the number of states
Then you use a nested for loop to update each of the values of the matrix according to your algorithm.
for c = 1:n
for r = 1:n
%Your code to calculate each value of the Matrix
M(r,c) = %desired probability
end
end
Also, there are some inbuilt functions for Markov Chain Modelling, you can have a look at:
