rlVectorEnv

R2026b

Create a vectorized environment from a scalar environment

Since R2026b

expand all in page

    Description

    Use rlVectorEnv to create a vectorized reinforcement learning environment by supplying an environment factory function that creates a scalar MATLAB® environment. Vectorized environments use built-in vectorization features to step and reset many environment instances in batch. This capability can improve hardware utilization and enable higher data throughput compared to non-vectorized (that is, single-instance, or scalar) environments. For agents that use large experience batches, training against a vectorized environment can be much faster than training against a corresponding scalar version of the environment. After creating the environment, call validateEnvironment to check that the environment is configured correctly.

    Note

    Vectorized environments created using rlFunctionVectorEnv typically perform better than equivalent environments created with rlVectorEnv.

    Creation

    Syntax

    venv = rlVectorEnv(envFactoryFcn)
    venv = rlFunctionVectorEnv(envFactoryFcn,NumEnv=N)

    Description

    venv = rlVectorEnv(envFactoryFcn) creates the vectorized reinforcement learning environment venv using the factory function envFactoryFcn.

    venv = rlFunctionVectorEnv(envFactoryFcn,NumEnv=N) also specifies the number of environment instances.

    example

    Input Arguments

    expand all

    Factory function for scalar environment, specified as a function handle.

    The factory function is a function that creates a scalar MATLAB environment. The function must have one input and one output, as illustrated by the following signature.

    senv = myEnvFactoryFcn(info)

    Here, info is a structure containing the fields EnvIdx (this is the index of the specific environment instance that is returned) and NumEnv (the total number of environments).

    You can use info.EnvIdx and info.NumEnv in the factory function to return an environment with different characteristics (as long as the observation and action specifications remain the same). For example, you can return an environment with a different parameter depending on the specific value of the environment instance index info.NumEnv

    Example: @(info) rlPredefinedEnv("CartPole-Continuous")

    Number of environment instances, specified as a positive integer. This argument sets the NumEnv property. If you omit this argument, the function uses the default value of 2.

    Example: NumEnv=128

    Properties

    expand all

    Number of environment instances, specified as a positive integer.

    Example: 128

    Object Functions

    rlFunctionVectorEnvCreate a vectorized reinforcement learning environment using your setup, reset, and step functions
    convertToScalarEnvConvert a vectorized environment to a scalar environment
    validateEnvironmentValidate custom reinforcement learning environment

    Examples

    collapse all

    Open Live Script

    For this example, use rlPredefinedEnv to create a cart-pole environment with a continuous action space, and then use rlVectorEnv to create a vectorized version of that environment.

    As a first argument, define in place an anonymous function that calls the environment factory function rlPredefinedEnv. This anonymous function returns the predefined cart-pole environment independently of info.

    Use the name-value NumEnv argument to create 128 environment instances.

    vcpenv = rlVectorEnv(@(info) rlPredefinedEnv("CartPole-Continuous"),NumEnv=128)
    vcpenv = 
  rlVectorEnv with properties:

    NumEnv: 128

    You can now create agents for vcpenv and train or simulate them as you would for any other environment.

    Alternatively, to returned environment instances that have different parameters, use a handle to a custom factory function, customFactoryFcn, defined at the end of the example. This function returns environment instances with varying mass ranging from 0.874 to 1.128. It uses the variable info.EnvIdx to create a different mass every time it is invoked. 

    n = 128;
vcpenv_massrange = rlVectorEnv(@(info) customFactoryFcn(info),NumEnv=n)
    vcpenv_massrange = 
  rlVectorEnv with properties:

    NumEnv: 128

    You can use this environment to train an agent that balances the pole with a policy that is robust to uncertainties in cart mass.

    Custom Factory Function

    function env = customFactoryFcn(info)
    env = rlPredefinedEnv("CartPole-Continuous");
    env.MassCart = 1 + 1e-3*(2*info.EnvIdx-info.NumEnv);
end

    Limitations

    • Setting the UseParallel training option to on is not supported with vectorized environments.

    • Training an agent using an evolution strategy (trainWithEvolutionStrategy) is not supported with vectorized environments.

    • Training Q-Learning, SARSA, LSPI, PG and AC agents is not supported with vectorized environments.

    • Training agents with actors or critics that use recursive neural networks is not supported with vectorized environments.

    Version History

    Introduced in R2026b

    See Also

    Functions

    Objects

    Topics