Controlling state transition using transition probability matrix in Markov chain?

3 次查看（过去 30 天）
chaaru datta
chaaru datta 2022-12-6
编辑： chaaru datta 2022-12-8
Hello all, I am working on Markov chain and in that I would like control the state transition using transition probability matrix (TPM).
The TPM in my case is 6x6 and is given as TPM = [0.6,0.4,0,0,0,0;0.3,0.4,0.3,0,0,0;0,0.3,0.4,0.3,0,0;0,0,0.3,0.4,0.3,0;0,0,0,0.3,0.4,0.3;0,0,0,0,0.4,0.6]. For clarity, let us denote the state at time t by the row and state at time t+1 by the column of TPM.
I understood that if my current state is 2 and if transition probability is 0.4 then from TPM my next state will also be 2.
But my query is how this condition of transition probability of value 0.4 is generated ?
Any help in this regard will be highly appreciated.

请先登录，再回答此问题。

回答（1 个）

Torsten
Torsten 2022-12-6
编辑：Torsten 2022-12-6
I understood that if my current state is 2 and if transition probability is 0.4 then from TPM my next state will also be 2.
That's wrong. If you are in state 2, the probability to change in state 1 is 0.3, to remain in state 2 is 0.4 and to change in state 3 is 0.3.
But my query is how this condition of transition probability of value 0.4 is generated ?
The transition probabilities are not generated, but they are fixed and calculated in advance depending on what your Markov chain tries to model.
  6 个评论
显示 4更早的评论
chaaru datta
chaaru datta 2022-12-7
Ok Thank you sir. But that again creates little confusion to me about Q-learning with Markov chain.
chaaru datta
chaaru datta 2022-12-8
编辑：chaaru datta 2022-12-8
My query is I am not getting about what value of next state should we put in Bellman equation (used in q-learning) if current state is , where Bellman equation is:

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Markov Chain Models 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by