Main Content

exponentialLearnRate

Exponential learning rate schedule

Since R2024b

expand all in page

    Description

    An exponential learning rate schedule decays the learning rate by a specified factor.

    Tip

    To easily use an exponential learning rate schedule with the default options, specify the LearnRateSchedule argument of the trainingOptions function as "exponential".

    Creation

    Description

    schedule = exponentialLearnRate creates an exponentialLearnRate object.

    This syntax is equivalent to setting the LearnRateSchedule argument of the trainingOptions function to "exponential".

    example

    schedule = exponentialLearnRate(Name=Value) specifies optional properties using one or more name-value arguments. For example, DropFactor=0.5 specifies halving of the learning rate.

    example

    Properties

    expand all

    Scaling factor used to drop the learning rate, specified as a scalar in the range (0,1).

    At each step, the software scales the learning rate using a factor of DropFactor^(1/Period), that is, by DropFactor after Period steps.

    If FrequencyUnit is "epoch", then the software updates the learning rate every epoch and each iteration of the epoch uses the same learning rate. If FrequencyUnit is "iteration", then the software updates the learning rate every iteration.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Number of steps before scaling the learning rate by DropFactor, specified as a positive integer.

    At each step, the software scales the learning rate using a factor of DropFactor^(1/Period), that is, by DropFactor after Period steps.

    If FrequencyUnit is "epoch", then the software updates the learning rate every epoch and each iteration of the epoch uses the same learning rate. If FrequencyUnit is "iteration", then the software updates the learning rate every iteration.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Frequency unit, specified as "epoch" or "iteration".

    If FrequencyUnit is "epoch", then the software updates the learning rate every epoch and each iteration of the epoch uses the same learning rate. If FrequencyUnit is "iteration", then the software updates the learning rate every iteration.

    This property is read-only.

    Number of steps to drop learning rate, specified as Inf.

    exponentialLearnRate objects are infinite learning rate schedules, so NumSteps is Inf.

    Data Types: double

    Examples

    collapse all

    Open Live Script

    Create an exponential learning rate schedule with the default settings.

    schedule = exponentialLearnRate
    schedule = 
  exponentialLearnRate with properties:

       DropFactor: 0.1000
           Period: 10
    FrequencyUnit: "epoch"
         NumSteps: Inf

    Specify this schedule as a training option.

    options = trainingOptions("adam",LearnRateSchedule=schedule);

    As an alternative to creating an exponential learning rate schedule object, to easily use an exponential learning rate schedule with the default options, specify the LearnRateSchedule argument of the trainingOptions function as "exponential".

    options = trainingOptions("adam",LearnRateSchedule="exponential");
    Open Live Script

    Create an exponential learning rate schedule that drops the learning rate by a factor of 2 every 100 iterations.

    schedule = exponentialLearnRate( ...
    DropFactor=0.5, ...
    Period=100, ...
    FrequencyUnit="iteration")
    schedule = 
  exponentialLearnRate with properties:

       DropFactor: 0.5000
           Period: 100
    FrequencyUnit: "iteration"
         NumSteps: Inf

    Specify this schedule as a training option.

    options = trainingOptions("adam",LearnRateSchedule=schedule);

    Algorithms

    expand all

    Version History

    Introduced in R2024b

    See Also

    | | | | | | |

    Topics