1. Epic GamesDeveloper
  3. FPPOTrainerTrainingSettings

FPPOTrainerTrainingSettings

Settings used for training with PPO

API > API/Plugins > API/Plugins/LearningTraining

References

   
Module LearningTraining
Header /Engine/Plugins/Experimental/LearningAgents/Source/LearningTraining/Public/LearningPPOTrainer.h
Include #include "LearningPPOTrainer.h"

Syntax

struct FPPOTrainerTrainingSettings

Remarks

Settings used for training with PPO

Variables

  Type Name Description
Public variable float ActionEntropyWeight Weighting used for the entropy bonus.
Public variable float ActionRegularizationWeight Weight used to regularize actions.Larger values will encourage exploration and smaller actions, but too large will cause noisy actions centered around zero.
Public variable float ActionSurrogateWeight Weight for the loss used to train the policy via the PPO surrogate objective.
Public variable float AdvantageMax The maximum advantage to allow.
Public variable float AdvantageMin The minimum advantage to allow.
Public variable bool bAdvantageNormalization When true, advantages are normalized.
Public variable bool bSaveSnapshots If to save snapshots of the trained networks every 1000 iterations.
Public variable bool bUseGradNormMaxClipping If true, uses gradient norm max clipping. Set this as True if training is unstable or leave as False if unused.
Public variable bool bUseTensorboard If to use TensorBoard for logging and tracking the training progress.
Public variable uint32 CriticBatchSize Batch size to use for training the critic. Large batch sizes are much more computationally efficient when training on the GPU.
Public variable uint32 CriticWarmupIterations Number of iterations of training to perform to warm-up the Critic.
Public variable ETrainerDevice Device Which device to use for training.
Public variable float DiscountFactor The discount factor causes future rewards to be scaled down so that the policy will favor near-term rewards over potentially uncertain long-term rewards.
Public variable float EpsilonClip Clipping ratio to apply to policy updates.
Public variable float GaeLambda This is used in the Generalized Advantage Estimation, where larger values will tend to assign more credit to recent actions.
Public variable float GradNormMax The maximum gradient norm to clip updates to.
Public variable uint32 IterationNum Number of iterations to train the network for.
Public variable uint32 IterationsPerGather Number of training iterations to perform per buffer of experience gathered.
Public variable float LearningRateCritic Learning rate of the critic network.
Public variable float LearningRateDecay Amount by which to multiply the learning rate every 1000 iterations.
Public variable float LearningRatePolicy Learning rate of the policy network. Typical values are between 0.001f and 0.0001f.
Public variable uint32 PolicyBatchSize Batch size to use for training the policy. Large batch sizes are much more computationally efficient when training on the GPU.
Public variable uint32 PolicyWindow The number of consecutive steps of observations and actions over which to train the policy.
Public variable float ReturnRegularizationWeight Weight used to regularize predicted returns. Encourages the critic not to over or under estimate returns.
Public variable uint32 Seed Random Seed to use for training.
Public variable int32 TrimEpisodeEndStepNum Number of steps to trim from the end of each episode during training.
Public variable int32 TrimEpisodeStartStepNum Number of steps to trim from the start of each episode during training.
Public variable float WeightDecay Amount of weight decay to apply to the network.

Ask questions and help your peers Developer Forums

Write your own tutorials or read those from others Learning Library

On this page