1. Epic GamesDeveloper
  3. FMLDeformerEditorModel

FMLDeformerEditorModel

The base class for the editor side of an UMLDeformerModel.

API > API/Plugins > API/Plugins/MLDeformerFrameworkEditor

Inheritance Hierarchy

References

   
Module MLDeformerFrameworkEditor
Header /Engine/Plugins/Animation/MLDeformer/MLDeformerFramework/Source/MLDeformerFrameworkEditor/Public/MLDeformerEditorModel.h
Include #include "MLDeformerEditorModel.h"

Syntax

class FMLDeformerEditorModel :  
    public TSharedFromThis< FMLDeformerEditorModel > ,  
    public FGCObject

Remarks

The base class for the editor side of an UMLDeformerModel. The editor model class handles most of the editor interactions, such as property changes, triggering the training process, creation of the viewport actors, and many more things. In most cases you will want to inherit your own editor model from either FMLDeformerGeomCacheEditorModel or FMLDeformerMorphModelEditorModel though. These classes will already implement a lot of base functionality in case you work with geometry caches as ground truth data etc.

Variables

  Type Name Description
Protected variable int32 ActiveTrainingInputAnimIndex The training input animation that is selected in the timeline.
Protected variable bool bDisplayFrames Display frame numbers?
Protected variable bool bIsReadyForTraining Are we ready for training? The training button in the editor will be enabled or disabled based on this on default.
Protected variable bool bIsScrubbingTimeline Are we currently scrubbing the timeline?
Protected variable bool bNeedsAssetReinit Set to true when on next tick we need to trigger an input assets changed event.
Protected variable bool bNeedToResampleInputOutputs Do we need to resample all input/output data?
Protected variable int32 CurrentTrainingFrame The current training frame.
Protected variable FMLDeformerEditorToolkit * Editor A pointer to the editor toolkit.
Protected variable TArray< FMLDeformerEditorActor * > EditorActors The set of actors that can appear inside the editor viewport.
Protected variable TObjectPtr< UMLDeformerInputInfo > EditorInputInfo The input info as currently setup in the editor.
Protected variable TObjectPtr< UMeshDeformer > HeatMapDeformerGraph The heatmap deformer graph.
Protected variable TObjectPtr< UMaterial > HeatMapMaterial The heatmap material.
Protected variable FDelegateHandle InputObjectModifiedHandle The delegate that handles when an object got modified (any object).
Protected variable FDelegateHandle InputObjectPropertyChangedHandle The delegate that handles when an object property got changed (any object).
Protected variable TSharedPtr< SMLDeformerInputWidget > InputWidget The inputs widget.
Protected variable TArray< FLODInfo > LODMappings The LOD information.
Protected variable TObjectPtr< UMLDeformerModel > Model The runtime model associated with this editor model.
Protected variable int32 NumTrainingFrames The total number of frames that can be used for training, based on the training input animations list.
Protected variable TRange< double > PlaybackRange The playback range of this model for each timeframe
Protected variable FDelegateHandle PostEditPropertyDelegateHandle The delegate handle to the post edit property event.
Protected variable TArray< TSharedPtr< FMLDeformerSampler > > Samplers A sampler for every input animation. These samplers are used to calculate the vertex deltas.
Protected variable FFrameTime ScrubPosition The current scrub position.
Protected variable TArray< FVector3f > SkinnedPositions A reusable array to store current LOD skinned vertex positions.
Protected variable TArray< TSharedPtr< FName > > VertexAttributeNames A list of vertex attribute names that can be used inside combobox UI elements.
Protected variable TRange< double > ViewRange The range we are currently viewing
Protected variable TRange< double > WorkingRange The working range of this model, encompassing the view range

Destructors

  Type Name Description
Public function Virtual   The destructor, which unregisters the instance of this editor model from the model registry.

Functions

  Type Name Description
Protected function void  
Public function void Add all animated bones to the bone include list.
Protected function void  
Public function void Add all animated curves to the curve include list.
Protected function void

AddCompareActor ( int32 ArrayIndex
)

  
Public function Virtual void

ApplyDebugActorTransforms ( const TArray< FTransform >& DebugActorComponentSpaceTransforms
)

 Apply the transforms of the actor we debug to the editor actors in our world.
Public function void Apply the transforms of the debug actor to the actors in the asset editor world.
Protected function Const void

CalcMeshNormals ( TArrayView< const FVector3f > VertexPositions,
TArrayView< const uint32 > IndexArray,
TArrayView< const int32 > VertexMap,
TArray< FVector3f >& OutNormals
)

 Calculate the normals for each vertex, given the triangle data and positions.
Protected function Const int32

CalcNumValidCompareActorsPriorTo ( int32 CompareActorIndex
)

  
Public function Virtual Const double Calculate the test mode timeline position.
Protected function Const double Calculate the current time line position offset, of the currently selected frame.
Public function Virtual Const double Get the current playback position of our training mode.
Public function Static void Change the skeletal mesh on a specific skeletal mesh component.
Public function void Check whether our training data frame number changed.
Public function Virtual void Clamp the current test frame index to be in a valid range.
Public function Virtual void Clamp the current training frame index to be in a valid range.
Public function Virtual void Clears the Persona preview scene. Resetting the preview mesh, preview mesh component etc.
Public function Virtual void Clears the world, which basically means it removes all the editor actors and engine actors from the editor world.
Public function Virtual void Clear both the world and preview scene. This calls ClearWorld and ClearPersonaPreviewScene.
Protected function void

CompressEngineMorphTargets ( FMorphTargetVertexInfoBuffers& OutMorphBuffers,
const TArray< UMorphTarget* >& MorphTargets,
int32 LOD,
float MorphErrorTolerance
)

 Compress morph targets into GPU based morph buffers.
Public function Virtual void

CopyBaseSettingsFromModel ( const FMLDeformerEditorModel* SourceEditorModel
)

 Copy the common property values from a source model into this model.
Public function Static float

CorrectedFrameTime ( int32 FrameNumber,
float FrameTimeForFrameNumber,
FFrameRate FrameRate
)

 Correct floating point errors that can cause issues when sampling animation using step timing.
Public function Virtual void

CreateActors ( const TSharedRef< IPersonaPreviewScene >& InPersonaPreviewScene
)

 Create the editor actors.
Public function Virtual Const FMLDeformerEditorActor * Create an actor in the editor viewport.
Protected function void

CreateEngineMorphTargets ( TArray< UMorphTarget* >& OutMorphTargets,
const TArray< FVector3f >& Deltas,
const FString& NamePrefix,
int32 LOD,
float DeltaThreshold,
bool bIncludeNormals,
const EMLDeformerMaskChannel MaskChannel,
bool bInvertMaskChannel
)

 Generate engine morph targets from a set of deltas.
Public function Virtual void Create the heat map assets for this model.
Public function Virtual void Create the heat map deformer graph for this model. This updates the HeatMapDeformerGraph member.
Public function Virtual void Create the heat map material for this model. This updates the HeatMapMaterial member.
Public function Virtual TSharedPtr< SMLDeformerInputWidget > Create the input widget.
Public function Virtual Const FMLDeformerSampler *  
Public function Virtual Const TSharedPtr< FMLDeformerSampler > Create the vertex delta sampler object.
Protected function Virtual void  
Public function Virtual void Create the compare actors for the testing mode.
Public function Virtual void Create the test ground truth actor.
Public function Virtual void Create the linear skinned test actor.
Public function Virtual void Create the test ML Deformed actor.
Public function Virtual void Create the training ground truth actor.
Public function Virtual void Create the linear skinned actor that is used as training base.
Protected function void Delete all editor actors.
Public function void

DrawMorphTarget ( FPrimitiveDrawInterface* PDI,
const TArray< FVector3f >& MorphDeltas,
float DeltaThreshold,
int32 MorphTargetIndex,
const FVector& DrawOffset
)

 Debug draw specific morph targets using lines and points.
Public function Virtual void Debug draw helpers inside the PIE viewport that highlight debuggable actors.
Public function Const FMLDeformerEditorActor *

FindEditorActor ( int32 ActorTypeID
)

 Find an editor actor with a specific ID, or return nullptr when not found.
Public function Const UMLDeformerComponent * Find the ML Deformer component of a given editor actor.
Public function Const TVertexAttributesConstRef< float >

FindVertexAttributes ( FName AttributeName
)

 Find the float based vertex attributes on the skeletal map that have a given name.
Public function Virtual void

GenerateNormalsForMorphTarget ( int32 LOD,
USkeletalMesh* SkelMesh,
int32 MorphTargetIndex,
TArrayView< const FVector3f > Deltas,
TArrayView< const FVector3f > BaseVertexPositions,
TArrayView< FVector3f > BaseNormals,
TArray< FVector3f >& OutDeltaNormals
)

 Generate the normals for a given morph target.
Public function Const int32  
Public function Const UAnimSequence * Get the currently active training input animation sequence, which is the one that the viewport is showing the timeline for in training mode.
Protected function Const const UAnimSequence * Get the animation sequence, either the test or training one, depending on in which mode we are in.
Public function Const FText Check whether the vertex counts of the skeletal mesh have changed compared to the one we have trained on.
Public function Const int32 Get the currently desired training frame number.
Public function Const FMLDeformerEditorToolkit * Get a pointer to the ML Deformer editor tookit. Cannot be a nullptr.
Public function Const const TArray< FMLDeformerEditorActor * > & Get all the created editor actors.
Public function Const UMLDeformerInputInfo * Get the editor's input info object.
Public function Const double Get the framerate specified by the anim sequence.
Public function Virtual Const FString Get the path to the heat map deformer graph asset.
Public function Virtual Const FString Get the path to the heat map material asset.
Public function Const FText

GetIncompatibleSkeletonErrorText ( const USkeletalMesh* InSkelMesh,
const UAnimSequence* InAnimSeq
)

 Check for incompatible skeletons between the skeletal mesh and anim sequence skeleton.
Public function Const FText Check for errors in the inputs.
Public function Const TSharedPtr< SMLDeformerInputWidget >  
Public function Const UMLDeformerModel * Get a pointer to the runtime ML Deformer model. Cannot really be a nullptr.
Public function Const int32 Get the number of curves on a specific skeletal mesh.
Public function Virtual Const int32 Get the total number of frames used for training, but limited by the maximum number of training frames we have setup.
Public function Virtual Const int32 Get the total number of frames in the test anim sequence.
Public function Virtual Const int32 Get the number of training frames.
Public function Virtual Const int32 Get the number of training input animations.
Public function Virtual Const FText Get the text that we should display as overlay on top of the asset editor viewport.
Public function Const TRange< FFrameNumber > Get the playback range of the model's data.
Public function Const bool Check whether we need to resample inputs and outputs.
Public function Const FMLDeformerSampler * Get the sampler we use to calculate vertex deltas when we are in training mdoe and enable deltas.
Public function Const FMLDeformerSampler *  
Public function Const FMLDeformerSampler * Get the vertex delta sampler for a given training input animation.
Public function Const float Get the current scrub time.
Public function Const FText Check to see if the skeletal mesh has to be reimported because it misses some newly added data.
Public function Virtual Const bool Check whether this model supports a per training input animation vertex mask.
Public function Const FText Check whether the vertex count in the target mesh has changed or not.
Public function Virtual Const int32

GetTestFrameAtTime ( double TimeInSeconds
)

 Get the test data frame number for a specific time offset in seconds.
Public function Virtual Const double

GetTestTimeAtFrame ( int32 FrameNumber
)

 Get the time in seconds, for a given frame in the test data.
Public function Const int32 Get the tick resolution we are displaying at.
Public function Const FFrameNumber Get the current scrub position.
Public function Const int32 Get how many ticks there are per frame.
Public function Virtual Const FMLDeformerEditorActor * Get the editor actor that defines the timeline play position.
Public function Virtual Const FString Get the onnx file name of the trained network.
Public function Virtual Const int32

GetTrainingFrameAtTime ( double TimeInSeconds
)

 Get the training data frame number for a specific time offset in seconds.
Public function Virtual Const FMLDeformerTrainingInputAnim * Get a given training input animation.
Public function Virtual Const double

GetTrainingTimeAtFrame ( int32 FrameNumber
)

 Get the time in seconds, for a given frame in the training data.
Public function Const const TArray< TSharedPtr< FName > > & Get the array of vertex attribute names that can be used for UI widget elements.
Public function Const FText Check whether the vertex map has changed or not.
Public function Const TRange< double > Get the current view range.Get the current view range
Protected function Const FMLDeformerEditorActor * Get the base actor depending on the currently active mode (train or test mode).
Public function Const TRange< double > Get the working range of the model's data.
Public function Const UWorld * Get a pointer to the world that our editor is displaying. Cannot be nullptr.
Public function void Handle the timeline changes when our model changes.
Public function void

HandleViewRangeChanged ( TRange< double > InRange
)

 Handle the view range being changed.
Public function void Handle the timeline changes required when changing the visualization mode, so when we switch between training and testing mode.
Public function void

HandleWorkingRangeChanged ( TRange< double > InRange
)

 Handle the working range being changed.
Public function Virtual void

Init ( const InitSettings& Settings
)

 Initialize the model.
Public function void Initialize the bone include list of the runtime model to the set of bones that are animated.
Public function void Initialize the curve include list of the runtime model to the set of curves that are animated.
Public function Virtual void Initialize the input information, which basically describes what the inputs to the neural network are going to be.
Public function void Mark the deltas to be updated on next Tick.
Protected function Virtual Const bool

IsAnimIndexValid ( int32 AnimIndex
)

  
Public function Virtual Const bool Check if this model is compatible with a specific deformer asset.
Public function Const bool Is the timeline displaying frames rather than seconds?
Protected function bool Perform some basic checks to see if the editor can be ready to train the model.
Public function Virtual Const bool Check whether we are playing an animation or not.
Public function Virtual Const bool Check whether our anim sequence is playing forward or backward.
Public function Const bool Check whether we are ready to train.
Public function Const bool Are we scrubbing on the timeline?
Public function Virtual Const bool Check whether our model is trained or not.
Public function Virtual Const UMeshDeformer * Load the default Optimus deformer graph to use on the skeletal mesh when using this model.
Public function Virtual Const bool Load the trained network from an onnx file.
Public function Virtual void Called whenever the actor to debug has been changed.
Public function Virtual void This is called when training data inputs change.
Public function Virtual void Executed when the maximum number of LOD levels changed.
Protected function Virtual void Called whenever an engine object is being modified.
Public function Virtual void Executed when the user presses the play button in the timeline.
Protected function void Executed when a property changes by a change in the UI. This internally calls OnPropertyChanged.
Public function Virtual void This is executed after the input assets have changed.
Public function Virtual void

OnPostTraining ( ETrainingResult TrainingResult,
bool bUsePartiallyTrainedWhenAborted
)

 This method is executed after training.
Public function Virtual void This is called just before we start the training process.
Public function Virtual void

OnPropertyChanged ( FPropertyChangedEvent& PropertyChangedEvent
)

 This is called whenever a property value changes in the UI.
Public function Virtual void

OnTimeSliderScrubPositionChanged ( double NewScrubTime,
bool bIsScrubbing
)

 This is executed when the timeline has been scrubbed.
Public function Virtual void

OnTrainingAborted ( bool bUsePartiallyTrainedData
)

 This is executed when the training process got aborted by the user.
Public function Virtual void This is triggered when the training frame is changed by the user.
Public function Virtual void Refresh the contents of the input widget. This includes the bone and curve list contents.
Public function Virtual void Call the SetupComponent on every ML Deformer component on our editor actors and update the compatibility status of the deformer components.
Protected function void  
Protected function void

RemoveCompareActor ( int32 ArrayIndex
)

  
Public function Virtual void

Render ( const FSceneView* View,
FViewport* Viewport,
FPrimitiveDrawInterface* PDI
)

 Render additional (debug) things in the viewport.
Public function Virtual void Sample the vertex deltas between the training base model and target model.
Public function void  
Public function Virtual void This makes sure that the deformer graph selected in the visualization settings is not a nullptr.
Public function void

SetDisplayFrames ( bool bDisplayFrames
)

 Set if frames are displayed.
Public function Virtual void

SetHeatMapMaterialEnabled ( bool bEnabled
)

 Specify whether the heat map material should be enabled or not.
Public function void

SetInputWidget ( TSharedPtr< SMLDeformerInputWidget > Widget
)

  
Public function void Set whether we need to resample inputs or not.
Public function void

SetScrubPosition ( FFrameNumber NewScrubPosition,
bool bIsScrubbing
)

  
Public function void

SetScrubPosition ( FFrameTime NewScrubPosition,
bool bIsScrubbing
)

  
Public function void

SetScrubPosition ( FFrameNumber NewScrubPosition
)

 Set the current scrub position.
Public function void

SetScrubPosition ( FFrameTime NewScrubPosition
)

 Set the current scrub position.
Public function Virtual void

SetTestFrame ( int32 FrameNumber
)

  
Public function Virtual void

SetTestFrame ( int32 FrameNumber,
bool bIsScrubbing
)

 Go to a given frame in the test data.
Protected function void Set the names of the anim sequences in the timeline.
Public function Virtual void

SetTrainingFrame ( int32 FrameNumber,
bool bIsScrubbing
)

 Go to a given frame in the training data.
Public function Virtual void

SetTrainingFrame ( int32 FrameNumber
)

  
Public function void

SetViewRange ( TRange< double > InRange
)

 Set the current view range.
Public function Virtual void

Tick ( FEditorViewportClient* ViewportClient,
float DeltaTime
)

 Tick the editor viewport.
Public function Virtual ETrainingResult

Train ()

 Launch the training.
Public function void

TriggerInputAssetChanged ( bool bRefreshVizSettings
)

 This will call OnInputAssetsChanged first, followed by OnPostInputAssetChanged.
Public function Virtual void Update LOD levels of the actors in editor world.
Public function Virtual void Update the transforms of all the editor actors.
Public function Virtual void Update the visibility status of the editor actors.
Protected function void  
Public function Virtual void This applies the right mesh deformer to the editor actors that have an ML Deformer component on them.
Public function void Update the editor's input info based on the current UI settings.
Protected function void Because of some technical reasons we cannot display a correct ground truth heat map at the moment while playing an animation.
Public function Virtual void Update the state that is used to decide whether the Train button in the UI is enabled or not.
Public function Virtual void Update the visibility of the labels, as well as their positions.
Protected function void  
Public function void Invalidate the memory usage, so it gets updated in the UI again.
Public function Virtual void Update the mesh offset factors, which is basically controlling where to place the actors in the scene.
Public function Virtual void Updates the number of available training frames as returned by GetNumTrainingFrames().
Public function Virtual void

UpdatePaintModePose ( bool bFullUpdate
)

 Update the character's pose in paint mode.
Public function Virtual void Update the persona preview scene's actor, skeletal mesh component, mesh and anim asset.
Protected function void Update the timeline related ranges, based on the length of the training or testing data.
Protected function void Update whether we force to use step interpolation on the ML Deformed model or not.
Public function Virtual void Update the playback speed of the test data.
Public function Virtual void Update the timeline's training input animation list.
Public function Virtual void This should update the list of available training devices.
Public function void Update the vertex attribute names array as returned by GetVertexAttributeNames().
Protected function void

ZeroDeltasByThreshold ( TArray< FVector3f >& Deltas,
float Threshold
)

 Zero all deltas with a length equal to, or smaller than the threshold value.

Overridden from FGCObject

  Type Name Description
Public function Virtual void Pure virtual that must be overloaded by the inheriting class.
Public function Virtual Const FString Overload this method to report a name for your referencer

Classes

  Type Name Description
Protected struct FLODInfo    
Public struct InitSettings The editor model initialization settings. This is used in the Init call.  

Ask questions and help your peers Developer Forums

Write your own tutorials or read those from others Learning Library

On this page