The correct procedure during training for the TD3 agent is to set the exploratory policy to 1.
When the exploratory policy is set to 1, it enables the agent to use the base agent exploration policy, which incorporates the exploratory model for noise parameters. This enables the agent to explore its action and observation spaces by introducing "stochastic" action selection, thus encouraging exploration during training.
When the exploratory policy is set to 0, it forces the agent to use the base agent greedy policy, resulting in "deterministic" action selection. In this case, the exploratory model (noise) will not influence the agent's actions during training, as the agent behaves deterministically selecting actions with maximum likelihood, and does not explore its action and observation spaces during deployment.
In the "rlTD3agent" documentation below, the "generatePolicyFunction" block is a function used to create a policy function for deployment.
Setting the "UseExplorationPolicy" property to true during training ensures agent behaves stochastically, while setting up the policy for deployment, setting the property to false ensures the agent behaves deterministically,
Hope this helps!
