how to simulate agent in environment and get the Observation from experience

Question

Shiyu Han 2022-3-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1670459-how-to-simulate-agent-in-environment-and-get-the-observation-from-experience

回答： Aditya 2024-1-16

load Agent01.mat

%%%Bulid environment

%%%Using custom functions

%The observation are position y,velocity ydot,angle velocity sidot

ObservationInfo=rlNumericSpec([6 1]);

ObservationInfo.Name='Vehicle States';

ObservationInfo.Description='X,Xdot,Y,Ydot,Si,Sidot';

%Create discrete actions

%Set the action(steering angle）from -pi/2 to pi/2

actions=-pi/2:pi/30:pi/2;

ActionInfo=rlFiniteSetSpec(actions);

ActionInfo.Name='Steering angle';

%Creating the custom environment using functions

env = rlFunctionEnv(ObservationInfo,ActionInfo,'myStepFunction','myResetFunction');

%load the saved agent

agent=saved_agent;

%simulation with saved agent and environment

%Configure simulation options

simOpts=rlSimulationOptions("MaxSteps",200,"NumSimulations",10);

%simulate the agent

experience=sim(env,agent,simOpts);

%The output structure experience records the observations collected from the environment,

% the action and reward, and other data collected during the simulation.

experience.Action

%Observations collected from the environment, returned as a structure with fields corresponding to the observations specified in the environment.

%Each field contains a timeseries of length N + 1, where N is the number of simulation steps

%Get the Observation State from a certain step

Obs = getsamples(experience.Observation.VehicleStates,1:10);

NextObs = getsamples(experience.Observation.VehicleStates,2:11);

I build an function environment and train an agent. Now I want to simulation the agent with experience=sim(env,agent,simOpts);

BUt why I can't get the Observation from the experience?

The error is "The middle dot '.' indexing produces a comma-separated list of 10 values, but it must produce a single value when followed by subsequent indexing operations."

Is that something wrong with simulation options? How to solve the problem? Thank you.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Aditya 2024-1-16

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1670459-how-to-simulate-agent-in-environment-and-get-the-observation-from-experience#answer_1390706

在 MATLAB Online 中打开

The error you're encountering isn't related to the simulation options but rather to the way you're trying to access the observations in the expirience structure.

The error message indicates that experience.Observation.VehicleStates is a comma-separated list containing multiple timeseries objects (one for each simulation if NumSimulations is set to 10), and you're trying to index into it as if it were a single timeseries object.

When you simulate the environment multiple times (NumSimulations is greater than 1), the experience structure will contain a cell array of experiences, one for each simulation. The correct way to access the observations for a specific simulation is to first index into the cell array to get the experience for that simulation, and then index into the timeseries.

If you want to access observations across all simulations, you will need to iterate through the cell array:

% Initialize cell arrays to hold observations from all simulations
ObsAll = cell(size(experience));
NextObsAll = cell(size(experience));
% Loop through each simulation's experience
for i = 1:length(experience)
    % Access the experience for the i-th simulation
    simExperience = experience(i);
    
    % Get the Observation State timeseries for the i-th simulation
    vehicleStatesTS = simExperience.Observation.VehicleStates;
    
    % Extract the observations for specific steps
    ObsAll{i} = getsamples(vehicleStatesTS, 1:10);
    NextObsAll{i} = getsamples(vehicleStatesTS, 2:11);
end