Main Content

rlOptimizer

Creates an optimizer object for actors and critics

Description

Create an optimizer object that updates the learnable parameters of an actor or critic in a custom training loop

example

algobj = rlOptimizer creates a default optimizer object. You can modify the object properties using dot notation.

example

algobj = rlOptimizer(algOptions) creates an optimizer object with properties specified by the optimizer options object algOptions.

Examples

collapse all

Use rlOprimizer to create a default optimizer algorithm object to use for the training of an actor or critic in a custom training loop.

myAlg = rlOptimizer
myAlg = 
  rlADAMOptimizer with properties:

           GradientDecayFactor: 0.9000
    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.0100
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

By default, the function returns an rlADAMOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.LearnRate = 0.1;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Use rlOprimizerOptions to create an optimizer option object. Specify the algorithm as "rmsprop" and set the learning rate to 0.2.

myOptions=rlOptimizerOptions( ...
    Algorithm="rmsprop", ...
    LearnRate=0.2);

Use rlOptimizer to create an optimizer algorithm object to use for the training of an actor or critic in a custom training loop. Specify the optimizer option set myOptions as input parameter.

myAlg=rlOptimizer(myOptions)
myAlg = 
  rlRMSPropOptimizer with properties:

    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.2000
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

The function returns an rlRMSPropOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.GradientThreshold = 2;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Input Arguments

collapse all

Algorithm options object, specified as an rlOptimizerOptions object.

Example: rlOptimizerOptions(Algorithm="sgdm",LearnRate=0.2)

Output Arguments

collapse all

Algorithm optimizer object, returned as an rlADAMOptimizer, rlSGDMOptimizer, or rlRMSPropOptimizer object. The runEpisode function uses the update method of the returned object to update the learnable parameter of an actor or critic.

Version History

Introduced in R2022a