getLearnableParameters

Obtain learnable parameter values from agent, function approximator, or policy object

collapse all in page

Syntax

params = getLearnableParameters(agent)

params = getLearnableParameters(fcnAppx)

params = getLearnableParameters(policy)

Description

Agent

params = getLearnableParameters(agent) returns the learnable parameter values from the agent object agent.

Actor or Critic

params = getLearnableParameters(fcnAppx) returns the learnable parameter values from the actor or critic function approximator object fcnAppx. This is equivalent to params=fcnAppx.Learnables.

example

Policy

params = getLearnableParameters(policy) returns the learnable parameters values from the policy object policy.

Examples

collapse all

Modify Critic Parameter Values

Open Live Script

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Compare DDPG Agent to LQR Controller.

load("DoubleIntegDDPG.mat","agent")

Obtain the critic from the agent.

critic = getCritic(agent);

For approximator objects, you can access the Learnables property using dot notation.

First, display the parameters.

critic.Learnables{1}

ans = 
  1x6 single dlarray

   -5.0017   -1.5513   -0.3424   -0.1116   -0.0506   -0.0047

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

critic.Learnables{1} = critic.Learnables{1}*2;

Display the new parameters.

critic.Learnables{1}

ans = 
  1x6 single dlarray

  -10.0034   -3.1026   -0.6848   -0.2232   -0.1011   -0.0094

Alternatively, you can use getLearnableParameters and setLearnableParameters.

First, obtain the learnable parameters from the critic.

params = getLearnableParameters(critic)

params=2×1 cell array
    {[-10.0034 -3.1026 -0.6848 -0.2232 -0.1011 -0.0094]}
    {[                                               0]}

Modify the parameter values. For this example, simply divide all of the parameters by 2.

modifiedParams = cellfun(@(x) x/2,params,"UniformOutput",false);

Set the parameter values of the critic to the new modified values.

critic = setLearnableParameters(critic,modifiedParams);

Set the critic in the agent to the new modified critic.

setCritic(agent,critic);

Display the new parameter values.

getLearnableParameters(getCritic(agent))

ans=2×1 cell array
    {[-5.0017 -1.5513 -0.3424 -0.1116 -0.0506 -0.0047]}
    {[                                              0]}

Modify Actor Parameter Values

Open Live Script

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Compare DDPG Agent to LQR Controller.

load("DoubleIntegDDPG.mat","agent")

Obtain the actor function approximator from the agent.

actor = getActor(agent);

For approximator objects, you can access the Learnables property using dot notation.

First, display the parameters.

actor.Learnables{1}

ans = 
  1x2 single dlarray

  -15.4663   -7.2746

Modify the parameter values. For this example, simply divide all of the parameters by 2.

actor.Learnables{1} = actor.Learnables{1}/2;

Display the new parameters.

actor.Learnables{1}

ans = 
  1x2 single dlarray

   -7.7331   -3.6373

Alternatively, you can use getLearnableParameters and setLearnableParameters.

Obtain the learnable parameters from the actor.

params = getLearnableParameters(actor)

params=2×1 cell array
    {[-7.7331 -3.6373]}
    {[              0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,"UniformOutput",false);

Set the parameter values of the actor to the new modified values.

actor = setLearnableParameters(actor,modifiedParams);

Set the actor in the agent to the new modified actor.

setActor(agent,actor);

Display the new parameter values.

getLearnableParameters(getActor(agent))

ans=2×1 cell array
    {[-15.4663 -7.2746]}
    {[               0]}

Input Arguments

collapse all

`agent` — Reinforcement learning agent
reinforcement learning agent object

Reinforcement learning agent, specified as one of the following objects:

rlQAgent
rlSARSAAgent
rlDQNAgent
rlPGAgent
rlDDPGAgent
rlTD3Agent
rlACAgent
rlSACAgent
rlPPOAgent
rlTRPOAgent
Custom agent — For more information, see Create Custom Reinforcement Learning Agents.

`fcnAppx` — Actor or critic function object
reinforcement learning function approximator object

Function approximator object, specified as one of the following:

rlValueFunction object — Value function critic
rlQValueFunction object — Q-value function critic
rlVectorQValueFunction object — Multi-output Q-value function critic with a discrete action space
rlContinuousDeterministicActor object — Deterministic policy actor with a continuous action space
rlDiscreteCategoricalActor object — Stochastic policy actor with a discrete action space
rlContinuousGaussianActor object — Stochastic policy actor with a continuous action space
rlContinuousDeterministicTransitionFunction object — Continuous deterministic transition function for neural network environments
rlContinuousGaussianTransitionFunction object — Continuous Gaussian transition function for neural network environments
rlContinuousDeterministicRewardFunction object — Continuous deterministic reward function for neural network environments
rlContinuousGaussianRewardFunction object — Continuous Gaussian reward function for neural network environments
rlIsDoneFunction object — IsDone function for neural network environments

To create an actor or critic function object, use one of the following methods.

Create a function object directly.
Obtain the existing critic from an agent using getCritic.
Obtain the existing actor from an agent using getActor.

`policy` — Reinforcement learning policy
`rlMaxQPolicy` | `rlEpsilonGreedyPolicy` | `rlDeterministicActorPolicy` | `rlAdditiveNoisePolicy` | `rlStochasticActorPolicy`

Reinforcement learning policy, specified as one of the following objects:

Output Arguments

collapse all

`params` — Learnable parameters
`rlValueFunction` object | `rlQValueFunction` object | `rlVectorQValueFunction` object | `rlContinuousDeterministicActor` object | `rlDiscreteCategoricalActor` object | `rlContinuousGaussianActor` object

Learnable parameter values for the function object, returned as a cell array. You can modify these parameter values and set them in the original agent or a different agent using the setLearnableParameters function.

Tips

You can also obtain and modify the learnable parameters function approximation objects such as actors and critics by accessing their Learnables property, using dot notation.

Version History

Introduced in R2019a

expand all

R2022a: `getLearnableParameters` now uses approximator objects instead of representation objects

Using representation objects to create actors and critics for reinforcement learning agents is no longer recommended. Therefore, getLearnableParameters now uses function approximator objects instead.

R2020a: `getLearnableParameterValues` is now `getLearnableParameters`

getLearnableParameterValues is now getLearnableParameters. To update your code, change the function name from getLearnableParameterValues to getLearnableParameters. The syntaxes are equivalent.

getLearnableParameters

Syntax

Description

Agent

Actor or Critic

Policy

Examples

Modify Critic Parameter Values

Modify Actor Parameter Values

Input Arguments

`agent` — Reinforcement learning agent
reinforcement learning agent object

`fcnAppx` — Actor or critic function object
reinforcement learning function approximator object

`policy` — Reinforcement learning policy
`rlMaxQPolicy` | `rlEpsilonGreedyPolicy` | `rlDeterministicActorPolicy` | `rlAdditiveNoisePolicy` | `rlStochasticActorPolicy`

Output Arguments

`params` — Learnable parameters
`rlValueFunction` object | `rlQValueFunction` object | `rlVectorQValueFunction` object | `rlContinuousDeterministicActor` object | `rlDiscreteCategoricalActor` object | `rlContinuousGaussianActor` object

Tips

Version History

R2022a: `getLearnableParameters` now uses approximator objects instead of representation objects

R2020a: `getLearnableParameterValues` is now `getLearnableParameters`

See Also

Functions

Objects

Topics

getLearnableParameters

Syntax

Description

Agent

Actor or Critic

Policy

Examples

Modify Critic Parameter Values

Modify Actor Parameter Values

Input Arguments

agent — Reinforcement learning agent reinforcement learning agent object

fcnAppx — Actor or critic function object reinforcement learning function approximator object

policy — Reinforcement learning policy rlMaxQPolicy | rlEpsilonGreedyPolicy | rlDeterministicActorPolicy | rlAdditiveNoisePolicy | rlStochasticActorPolicy

Output Arguments

params — Learnable parameters rlValueFunction object | rlQValueFunction object | rlVectorQValueFunction object | rlContinuousDeterministicActor object | rlDiscreteCategoricalActor object | rlContinuousGaussianActor object

Tips

Version History

R2022a: getLearnableParameters now uses approximator objects instead of representation objects

R2020a: getLearnableParameterValues is now getLearnableParameters

See Also

Functions

Objects

Topics

`agent` — Reinforcement learning agent
reinforcement learning agent object

`fcnAppx` — Actor or critic function object
reinforcement learning function approximator object

`policy` — Reinforcement learning policy
`rlMaxQPolicy` | `rlEpsilonGreedyPolicy` | `rlDeterministicActorPolicy` | `rlAdditiveNoisePolicy` | `rlStochasticActorPolicy`

`params` — Learnable parameters
`rlValueFunction` object | `rlQValueFunction` object | `rlVectorQValueFunction` object | `rlContinuousDeterministicActor` object | `rlDiscreteCategoricalActor` object | `rlContinuousGaussianActor` object

R2022a: `getLearnableParameters` now uses approximator objects instead of representation objects

R2020a: `getLearnableParameterValues` is now `getLearnableParameters`