unreal.DetailPoseModel

class unreal.DetailPoseModel(outer: Object | None = None, name: Name | str = 'None')

Bases: NeuralMorphModel

The detail pose model for the ML Deformer. This model is inherited from the Neural Morph Model, but adds additional morph targets on top of this. The additional morph targets that are generated are morph targets that bring it to the ground truth as seen during training at specific poses. These special poses are called Detail Poses. We basically look at the deltas that we predict using the neural morph model, and put the remaining error between that and the ground truth into the morph target at the given pose. There can be multiple detail poses, and each of them generates a new morph target.

C++ Source:

  • Plugin: DetailPoseModel

  • Module: DetailPoseModel

  • File: DetailPoseModel.h

Editor Properties: (see get_editor_property/set_editor_property)

  • alignment_transform (Transform): [Read-Write] The transform that aligns the Geometry Cache to the SkeletalMesh. This will mostly apply some scale and a rotation, but no translation.

  • anim_sequence (AnimSequence): [Read-Write] The animation sequence to apply to the base mesh. This has to match the animation of the target mesh’s geometry cache. Internally we force the Interpolation property for this motion to be “Step”. deprecated: Use the training input anims instead.

  • batch_size (int32): [Read-Write] The number of frames per batch when training the model.

  • blend_speed (float): [Read-Write] The speed at which the detail poses are blend in. Higher values make it blend in faster. A value of 0.0 would disable the detail poses from being calculated. A value of 1.0 would disable blending and instantly switch the active detail pose weight.

  • bone_group_mask_infos (Map[Name, NeuralMorphMaskInfo]): [Read-Write] deprecated: This property has been deprecated, please use BoneGroupMaskInfoMap instead.

  • bone_mask_infos (Map[Name, NeuralMorphMaskInfo]): [Read-Write] deprecated: This property has been deprecated, please use BoneMaskInfoMap instead.

  • clamp_morph_weights (bool): [Read-Write] Should we enable morph target weight clamping? The minimum and maximum values that it will be clamped against will be the min/max morph target weight values that have been seen when running the training dataset through the network. The advantage of clamping is that it can make deformations more stable when we have input poses that have not been seen during training. We basically prevent the weights from ‘exploding’ and getting very large values, which could make the mesh look very bad.

  • delta_cutoff_length (float): [Read-Write] deprecated: This property has been removed and isn’t used anymore.

  • detail_poses_anim_sequence (AnimSequence): [Read-Write] For certain poses the ML Deformer might not reconstruct important details such as specific cloth folding patterns. To improve the results near these important poses, you can provide a set of these poses. The poses are provided similar to the training input data, with an animation sequence and geometry cache pair where each frame contains a pose you want to preserve details in. The more poses you add, the higher the memory usage. Each pose will generate an additional morph target. If you do not add any detail poses, the model will behave the same as the Neural Morph Model in global mode. Please keep the frame rate and frame count the same for both the anim sequence and geometry cache.

  • detail_poses_geom_cache (GeometryCache): [Read-Write] For certain poses the ML Deformer might not reconstruct important details such as specific cloth folding patterns. To improve the results near these important poses, you can provide a set of these poses. The poses are provided similar to the training input data, with an animation sequence and geometry cache pair where each frame contains a pose you want to preserve details in. The more poses you add, the higher the memory usage. Each pose will generate an additional morph target. If you do not add any detail poses, the model will behave the same as the Neural Morph Model in global mode. Please keep the frame rate and frame count the same for both the anim sequence and geometry cache.

  • enable_bone_masks (bool): [Read-Write] Enable the use of per bone and bone group masks. When enabled, an influence mask is generated per bone based on skinning info. This will enforce deformations localized to the area around the joint. The benefit of enabling this can be reduced GPU memory footprint, faster GPU performance and more localized deformations. If deformations do not happen near the joint then this enabling this setting can lead to those deformations possibly not being captured.

  • geometry_cache (GeometryCache): [Read-Write] The geometry cache that represents the target deformations. deprecated: Use the training input anims instead.

  • global_num_hidden_layers (int32): [Read-Write] The number of hidden layers that the neural network model will have.nHigher numbers will slow down performance but can deal with more complex deformations.

  • global_num_morph_targets (int32): [Read-Write] The number of morph targets to generate in total. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

  • global_num_neurons_per_layer (int32): [Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations.

  • include_bones (bool): [Read-Write] deprecated: This property has been removed and isn’t used anymore.

  • include_curves (bool): [Read-Write] deprecated: This property has been removed and isn’t used anymore.

  • include_normals (bool): [Read-Write] Include vertex normals in the morph targets? The advantage of this can be that it is higher performance than recomputing the normals. The disadvantage is it can result in lower quality and uses more memory for the stored morph targets. In most cases you want this unchecked and calculate the normals using a deformer graph or using the skeletal mesh’s tangent recompute settings in the section details.

  • invert_mask_channel (bool): [Read-Write] Enable this if you want to invert the mask channel values. For example if you painted the neck seam vertices in red, and you wish the vertices that got painted to NOT move, you have to invert the mask. On default you paint areas where the deformer should be active. If you enable the invert option, you paint areas where the deformer will not be active.

  • learning_rate (float): [Read-Write] The learning rate used during the model training.

  • local_num_hidden_layers (int32): [Read-Write] The number of hidden layers that the neural network model will have. Higher numbers will slow down performance but can deal with more complex deformations. For the local model you most likely want to stick with a value of one or two.

  • local_num_morph_targets_per_bone (int32): [Read-Write] The number of morph targets to generate per bone, curve or group. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

  • local_num_neurons_per_layer (int32): [Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations. For the local mode you probably want to keep this around the same value as the number of morph targets per bone.

  • mask_channel (MLDeformerMaskChannel): [Read-Write] The channel data that represents the delta mask multipliers. You can use this feather out influence of the ML Deformer in specific areas, such as neck line seams, where the head mesh connects with the body. The painted vertex color values will be like a weight multiplier on the ML deformer deltas applied to that vertex. You can invert the mask as well.

  • max_num_lo_ds (int32): [Read-Write] How many Skeletal Mesh LOD levels should we generate MLD lods for at most? Some examples: A value of 1 means we only store one LOD, which means LOD0. A value of 2 means we support this ML Deformer on LOD0 and LOD1. A value of 3 means we support this ML Deformer on LOD0 and LOD1 and LOD2. We never generate more LOD levels for the ML Deformer than number of LOD levels in the Skeletal Mesh, so if this value is set to 100, while the Skeletal Mesh has only 4 LOD levels, we will only generate and store 4 ML Deformer LODs. The default value of 1 means we do not support this ML Deformer at LOD levels other than LOD0. When cooking, the console variable “sg.MLDeformer.MaxLODLevelsOnCook” can be used to set the maximum value per device or platform.

  • max_training_frames (int32): [Read-Write] The maximum numer of training frames (samples) to train on. Use this to train on a sub-section of your full training data.

  • mode (NeuralMorphMode): [Read-Write] The mode that the neural network will operate in. Local mode means there is one tiny network per bone, while global mode has one network for all bones together. The advantage of local mode is that it has higher performance, while global mode might result in better deformations.

  • morph_compression_level (float): [Read-Write] The morph target compression level. Higher values result in larger compression, but could result in visual artifacts. Most of the times this is a value between 1 and 50. It is best to first try increasing the “Delta Zero Threshold” property as high as you can though. Once that is set to the highest acceptable value, try increasing this morph comrpession level as high as visually acceptable. On the left side in the MLD asset editor you can see the estimated GPU Memory Usage. That value should change when you modify this property.

  • morph_delta_zero_threshold (float): [Read-Write] Morph target delta values that are smaller than or equal to this threshold will be zeroed out. This essentially removes small deltas from morph targets, which will lower the memory usage at runtime, however when set too high it can also introduce visual artifacts. A value of 0 will result in the highest quality morph targets, at the cost of higher runtime memory usage. On the left side in the MLD asset editor you can see the estimated GPU Memory Usage. That value should change when you modify this property.

  • num_iterations (int32): [Read-Write] The number of iterations to train the model for. If you are quickly iterating then around 1000 to 3000 iterations should be enough. If you want to generate final assets you might want to use a higher number of iterations, like 10k to 100k or even up to one million. Once the loss doesn’t go down anymore, you know that more iterations most likely won’t help much.

  • rbf_range (float): [Read-Write] The range to blend detail poses. Larger values will blend more detail poses together but also result in slower GPU performance.

  • regularization_factor (float): [Read-Write] The regularization factor. Higher values can help generate more sparse morph targets, but can also lead to visual artifacts. A value of 0 disables the regularization, and gives the highest quality, at the cost of higher runtime memory usage.

  • skeletal_mesh (SkeletalMesh): [Read-Write] The skeletal mesh that represents the linear skinned mesh.

  • skinning_mode (MLDeformerSkinningMode): [Read-Write] The base skinning mode to use. The learned deltas will be applied on top of this basis. Please keep in mind that you use the same type of skinning on your character, otherwise the applied deltas are wrong. If you train using Dual Quaternions you should also run a Dual Quaternion skinning deformer before applying the ML Deltas on top.

  • smooth_loss_beta (float): [Read-Write] The beta parameter in the smooth L1 loss function, which describes below which absolute error to use a squared term. If the error is above or equal to this beta value, it will use the L1 loss. This is a non-negative value, where 0 makes it behave exactly the same as an L1 loss. The value entered represents how many cm of error to allow before an L1 loss is used. Typically higher values give smoother results. If you see some noise in the trained results, even with large amount of samples and iterations, try increasing this value.

  • training_device (str): [Read-Write] The device used for training. On default it will init to Cuda’s preferred device, or Cpu if no such device present.

  • training_input_anims (Array[MLDeformerGeomCacheTrainingInputAnim]): [Read-Write] The training animation inputs.

  • use_rbf (bool): [Read-Write] Use RBF (Radial Basis Functions) to interpolate? This will produce higher quality blends between detail poses, at the cost of runtime CPU performance.

  • vertex_attribute_name (Name): [Read-Write] The global vertex attribute attribute name. This is an attribute on the skeletal mesh, which can be created using the skeletal mesh editor.