The "Reinforcement Learning Toolbox" does not natively provide a mechanism for passing data from one episode to the next through the environment's "ResetFcn" since the "ResetFcn" is stateless by default. A workaround would be to implement the adaptive domain randomization by using a persistent variable or a global stateful mechanism outside the environment that stores the most recent reward and adjusts parameters accordingly. This is achieved by storing the reward of the previous episode and using it inside your "ResetFcn" to modify the mass range.
Additionally, a better approach would be to create a custom class that tracks performance between episodes. Refer to the following steps to understand the general approach:
- Create a Custom Tracker Class
classdef DomainRandomizationManager < handle
properties
rewardHistory = [];
NominalValue = 10; % example nominal mass
end
methods
function updateReward(obj, reward)
obj.rewardHistory(end+1) = reward;
end
function [Mmin, Mmax] = getRandomizationRange(obj)
if isempty(obj.rewardHistory)
rangePercent = 0.1;
else
avgReward = mean(obj.rewardHistory(max(end-4,1):end)); % moving average
if avgReward < 100
rangePercent = 0.1;
else
rangePercent = 0.2;
end
end
Mmin = obj.NominalValue * (1 - rangePercent);
Mmax = obj.NominalValue * (1 + rangePercent);
end
end
end
- Use it in the Reset Function
- Hook into the Training Loop. MATLAB’s built-in "train" function does not allow easy feedback between episodes. So, for full flexibility, switch to a custom training loop:
global DRM;
DRM = DomainRandomizationManager();
for ep = 1:maxepisodes
experience = [];
totalReward = 0;
% reset env — ResetFcn uses DRM internally
initialObs = reset(env);
for t = 1:maxsteps
action = getAction(agent, initialObs);
[nextObs, reward, isDone, ~] = step(env, action);
totalReward = totalReward + reward;
experience = [experience; {initialObs, action, reward, nextObs}];
initialObs = nextObs;
if isDone
break;
end
end
DRM.updateReward(totalReward); % update the reward history
% Train the agent using collected experience
agent = train(agent, experience); % or use rlReplayBuffer if needed
end
Refer to the following MATLAB documentation for more information:
