Training is getting stuck halfway.

2 次查看（过去 30 天）
Abhishek
Abhishek 2023-5-17
回答： Emmanouil Tzorakoleftherakis 2023-5-23
Hi Community,
I have a Battery system where agent should do the trading based on the price of electricty at an hour. The profit is to be maximised. the price data for 8760 hours is loaded as a .mat file. The environment is created successfully but while training using MATLAB RL designer app, using PPO algorithm , the trainig is getting stuck after few episodes throwing the following error.
Error in executing callback registered with ViewModel: Error using rl.internal.app.tool.TrainingSession/qeStopTimer Operands to the logical AND (&&) and OR (||) operators must be convertible to logical scalar values. Use the ANY or ALL functions to reduce operands to logical scalar values.
Please guide on the same.
The environment is successfully being created with the following details
ans =
bessim with properties:
BatteryLevel: 4000
power: 0
EnvName: 'EnergyTrading'
EnergyPricedata: [8760×1 double]
Capacity: 8000
Pmax: 2000
hoursperyear: 8760
current_hour: 1
Ts: 1
State: [3×1 double]
The code is as follows;
classdef bessim < rl.env.MATLABEnvironment
%BESSIM: Template for defining custom environment in MATLAB.
%% Properties (set properties' attributes accordingly)
properties
% Specify and initialize environment's necessary properties
BatteryLevel % Battery level (kWh)
power % Grid power (kW)
EnvName= 'EnergyTrading';
EnergyPricedata
Capacity = 8000;
Pmax = 2000 ;
hoursperyear = 8760 ;
current_hour=1;
Ts=1;
end
properties
State = zeros(3,1) ;
end
properties(Access = protected)
IsDone = false ;
end
%% Necessary Methods
methods
% Constructor method creates an instance of the environment
function this = bessim(~)
% Initialize Observation settings
ObservationInfo = rlNumericSpec([3 1]);
ObservationInfo.Name = 'BatteryLevel';
ObservationInfo.LowerLimit = [0; -2000; 0];
ObservationInfo.UpperLimit = [8000; 2000; inf];
% Initialize Action settings
ActionInfo = rlNumericSpec([1 1]);
ActionInfo.Name = 'Charging Power';
ActionInfo.LowerLimit = 0;
ActionInfo.UpperLimit = 2000;
% Call the parent constructor to create the environment
this = this@rl.env.MATLABEnvironment(ObservationInfo, ActionInfo);
% Initialize property values and load energy price data
this.BatteryLevel = this.Capacity/2;
this.power = 0;
data1 = load('Energypriceprofile.mat');
this.EnergyPricedata = data1.Energypriceprofile; % Assuming the variable in the mat file is 'energypriceprofile'
this.current_hour = 1;
end
% Apply system dynamics and simulates the environment with the
% given action for one step.
function [Observation,Reward,IsDone,LoggedSignals] = step(this,Action)
LoggedSignals = [];
% Get action
ChargingPower = Action;
DischargingPower = -Action;
% Get current cost data
CurrentPrice=this.EnergyPricedata(this.current_hour);
NetProfit = DischargingPower * CurrentPrice - ChargingPower * CurrentPrice;
% Calculate power balance
% Update battery level
if NetProfit >= 0
% Charge the battery
ChargingTime = min((this.Capacity - this.BatteryLevel) / ChargingPower, this.Ts);
this.BatteryLevel = this.BatteryLevel + ChargingPower * ChargingTime;
else
% Discharge the battery
DischargingTime = min(this.BatteryLevel / abs(DischargingPower), this.Ts);
this.BatteryLevel = this.BatteryLevel + DischargingPower * DischargingTime;
end
% Calculate cost and revenue
% Update grid power
this.power = ChargingPower - DischargingPower;
% Set observation
Observation = [this.BatteryLevel; this.power; CurrentPrice];
% Check terminal condition (none in this example)
IsDone = false;
% Increment current hour
this.current_hour = this.current_hour + 1;
% Set reward
Reward = NetProfit;
end
function InitialObservation = reset(this)
this.BatteryLevel = this.Capacity/2;
this.State = [this.BatteryLevel] ;
InitialObservation = [this.BatteryLevel; this.power; this.EnergyPricedata(this.current_hour)];
% (optional) use notifyEnvUpdated to signal that the
% environment has been updated (e.g. to update visualization)
notifyEnvUpdated(this);
end
end

请先登录，再回答此问题。

回答（1 个）

Emmanouil Tzorakoleftherakis
Hi,
The error message seems to be longer than what you pasted. It appears there is an indexing error in the step method. Did not check in details, but the following line seems like a good place to start
CurrentPrice=this.EnergyPricedata(this.current_hour);

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by