Definition of transitions and rewards in a MDP environment for a reinforcement learning problem.
2 次查看(过去 30 天)
显示 更早的评论
Good morning,
I have an environment with 50 states and 5 actions. From state 1, I transition to states 2 to 25 with action 1; from state 1, I transition to states 26 to 49 with action 2; from states 2 to 25, I transition to state 50 with action 3; from states 26 to 49, I transition to state 50 with action 4. I define the matrices MDP.T as follows:
MDP.T(1,2:25,1) = 1/24;
MDP.T(1,26:49,2) = 1/24;
MDP.T(2:25,50,3) = 1;
MDP.T(26:49,50,4) = 1;
MDP.T(50,50,5) = 0;
The code works only if I consider also these other transitions, which are not feasible in my example.
MDP.T(1,1,3) = 1;
MDP.T(1,1,4) = 1;
MDP.T(2:25,2:25,1) = 1/24;
MDP.T(2:25,2:25,2) = 1/24;
MDP.T(2:25,2:25,4) = 1/24;
MDP.T(2:25,2:25,5) = 1/24;
MDP.T(26:49,26:49,1) = 1/24;
MDP.T(26:49,26:49,2) = 1/24;
MDP.T(26:49,26:49,3) = 1/24;
MDP.T(26:49,26:49,5) = 1/24;
May you help me to understand how to define only feabile transitions?
Thank you in advance
采纳的回答
arushi
2024-1-18
Hi Bianca,
As per my understanding, it seems that you want to define a transition matrix for an MDP with 50 states and 5 actions, where certain transitions are possible based on the current state and the action taken. The code does not work if you do not consider the other transitions. In some MDP implementations, especially when using certain toolboxes or libraries, it's necessary to define transitions for all state-action pairs, even if some actions are not applicable in certain states. This is typically done to avoid undefined behavior in the MDP solver or to comply with the requirements of the specific MDP framework being used.
To handle this in the code, you can assign a self-transition (transition back to the same state) with a probability of 1 for actions that are not applicable in each state. This effectively means that if you take an infeasible action in a state, you stay in the same state with certainty.
Hope this helps.
0 个评论
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 AI for Wireless 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!