Hi Jacob,
For computing
against time for each of the states, you can:
against time for each of the states, you can:- Compute powers of the transition matrix
- Extract the diagonal elements
- Compute the cumulative sum for each state. (Can maintain a row vector corresponding to each state, length = no of steps taken)
% Define the transition matrix
T = [2/3, 0, 1/3;
1/4, 3/4, 0;
1/3, 0, 2/3];
% Number of steps (define)
n_max = 5;
% Initialize R_i(n) for each state (1,2,3)
R = zeros(3, n_max);
Tk = T; % Start with T^1
for n = 1:n_max
diag_elements = diag(Tk); % Extract diagonal elements from (T^n)_ii
if n == 1
R(:, n) = diag_elements;
else
R(:, n) = R(:, n-1) + diag_elements;
end
% Update matrix power T^n -> T^(n+1)
Tk = Tk * T;
end
% Plot
figure;
hold on;
plot(1:n_max, R(1, :), 'r', 'LineWidth', 2);
plot(1:n_max, R(2, :), 'g', 'LineWidth', 2);
plot(1:n_max, R(3, :), 'b', 'LineWidth', 2);
hold off;
xlabel('n (Time Step)');
ylabel('R_i(n)');
title('Return Frequency R_i(n) for Each State');
legend('State 1', 'State 2', 'State 3');
grid on;
Additionally you can refer to the following functionalities which are used for the above computation and plotting
Further you may want review the MATLAB onramp course which can be useful for getting good hands-on experience in MATLAB: https://matlabacademy.mathworks.com/details/matlab-onramp/gettingstarted
Hope this helps!
