Hi Laura,
From what I can gather, you are looking for an implementation of Bellman equation with 2 independent variables in MATLAB.
V(a, b) = max{u(a, b) + βV(a’, b’)};
Here’s a snippet of code that tries to implement this Bellman equation in MATLAB:
beta = 0.9; % Discount rate
A = linspace(0, 10, 50);
B = linspace(0, 10, 50);
% directions for future states (You can add or remove values from here)
dir = [1 0; -1 0; 0 1; 0 -1];
Here, I have created 2 example matrices (A and B) of size 1x50 each. For this example, I have added 4 directions to dir. This will help us to traverse the reward function V.
function V = bellmanEQwith2Vars(beta, A, B, dir)
% Sample utility function
u = @(x, y) x + y;
V = zeros(length(A) + 2, length(B) + 2);
% For this example we are going to do 10 iterations
for iter = 1:10
for i = 1:length(A)
for j = 1:length(B)
for k = 1:length(dir)
V(i + 1, j + 1) = max(V(i + 1, j + 1), u(A(i), B(j)) ...
+ beta * V(i + dir(k, 1) + 1, j + dir(k, 2) + 1));
end
end
end
end
end
Change the utility function(u) according to your needs.
Please use the following link to learn more about bellman equations and dynamic programming:
I hope this help, thanks!
