Dear all,
While in environments with Discrete action spaces, the procedure works well, in Environments with CONTINUOUS action spaces gives an error in the step function inside the needed wrapper class, in the second line.
The lines that throws the error are:
ActionSingle = single(Action);
result = this.open_env.step(ActionSingle);
The error I get is :
Error using rl.train.SeriesTrainer/run
There was an error executing the environment's step method.
Error using continuous_mountain_car>step
Python Error: TypeError: 'float' object is not subscriptable
I think this error has something to do with the integration of Python ndarrays in Matlab.
The same issue is reported here:
https://es.mathworks.com/matlabcentral/answers/568278-environment-for-reinforcement-learning-project?s_tid=srchtitle
How can I solve this problem so that I can train environments with continuous action spaces?
Thank you in advance,
Alberto
CODE FOR THE WRAPPER CLASS
classdef MountainCarContinuousEnv < rl.env.MATLABEnvironment
open_env = py.gymnasium.make("MountainCarContinuous-v0");
function this = MountainCarContinuousEnv()
ObservationInfo = rlNumericSpec([2 1],'LowerLimit',[-1.2 -0.07]', 'UpperLimit', [0.6 0.07]');
ObservationInfo.Name = 'MountainCarObservation';
ObservationInfo.Description = 'Position, Velocity';
ActionInfo = rlNumericSpec([1 1],'LowerLimit',-1, 'UpperLimit', 1);
ActionInfo.Name = 'DirectionalForce';
this = this@rl.env.MATLABEnvironment(ObservationInfo, ActionInfo);
function [Observation,Reward,IsDone,LoggedSignals] = step(this,Action)
ActionSingle = single(Action);
result = this.open_env.step(ActionSingle);
Observation = double(result{1})';
function InitialObservation = reset(this)
result = this.open_env.reset();
InitialObservation = double(result{1})';
function result =render(this)
result = this.open_env.render();
function result =close(this)
result = this.open_env.close();
