Navigation
API > API/Plugins > API/Plugins/MetaHumanPerformance
MetaHuman Performance Asset
Produces an Animation Sequence for MetaHuman Control Rig by tracking facial expressions in video-footage from a Capture Source, imported through Capture Manager, using a SkeletalMesh obtained through MetaHuman Identity asset toolkit.
| Name | UMetaHumanPerformance |
| Type | class |
| Header File | /Engine/Plugins/MetaHuman/MetaHumanAnimator/Source/MetaHumanPerformance/Public/MetaHumanPerformance.h |
| Include Path | #include "MetaHumanPerformance.h" |
Syntax
UCLASS (BlueprintType)
class UMetaHumanPerformance : public UObject
Inheritance Hierarchy
- UObjectBase → UObjectBaseUtility → UObject → UMetaHumanPerformance
Constructors
| Name | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|
UMetaHumanPerformance() |
MetaHumanPerformance.h |
Classes
| Name | Remarks |
|---|---|
| FOnProcessingFinishedDynamic |
Typedefs
| Name | Type | Remarks | Include Path |
|---|---|---|---|
| FOnControlRigClassChanged | TMulticastDelegate_OneParam< void, TSubclassOf< class UControlRig > InControlRig > | MetaHumanPerformance.h | |
| FOnDataInputTypeChanged | TMulticastDelegate_OneParam< void, EDataInputType InProcessingType > | Event delegates. | MetaHumanPerformance.h |
| FOnDepthChanged | TMulticastDelegate_FourParams< void, float InDepthDataNear, float InDepthDataFar, float InDepthMeshNear, float InDepthMeshFar > | MetaHumanPerformance.h | |
| FOnExcludedFramesChanged | TMulticastDelegate_NoParams< void > | MetaHumanPerformance.h | |
| FOnFrameProcessed | TMulticastDelegate_OneParam< void, int32 InFrame > | MetaHumanPerformance.h | |
| FOnFrameRangeChanged | TMulticastDelegate_TwoParams< void, int32 InStartFrame, int32 InEndFrame > | MetaHumanPerformance.h | |
| FOnHeadMovementModeChanged | TMulticastDelegate_OneParam< void, EPerformanceHeadMovementMode InHeadMovementMode > | MetaHumanPerformance.h | |
| FOnHeadMovementReferenceFrameChanged | TMulticastDelegate_TwoParams< void, bool bInAutoChooseHeadMovementReferenceFrame, uint32 InHeadMovementReferenceFrame > | MetaHumanPerformance.h | |
| FOnIdentityChanged | TMulticastDelegate_OneParam< void, class UMetaHumanIdentity *InIdentity > | MetaHumanPerformance.h | |
| FOnNeutralPoseCalibrationChanged | TMulticastDelegate_NoParams< void > | MetaHumanPerformance.h | |
| FOnProcessingFinished | TMulticastDelegate_OneParam< void, TSharedPtr< const UE::MetaHuman::Pipeline::FPipelineData > InPipelineData > | MetaHumanPerformance.h | |
| FOnRealtimeAudioChanged | TMulticastDelegate_OneParam< void, bool bInRealtimeAudio > | MetaHumanPerformance.h | |
| FOnSourceDataChanged | TMulticastDelegate_ThreeParams< void, class UFootageCaptureData *InFootageCaptureData, class USoundWave *InAudio, bool InResetRanges > | MetaHumanPerformance.h | |
| FOnStage1ProcessingFinished | TMulticastDelegate_NoParams< void > | MetaHumanPerformance.h | |
| FOnVisualizeMeshChanged | TMulticastDelegate_OneParam< void, class USkeletalMesh *InVisualizeMesh > | MetaHumanPerformance.h |
Constants
| Name | Type | Remarks | Include Path |
|---|---|---|---|
| CurrentlyProcessedPerformance | TWeakObjectPtr< UMetaHumanPerformance > | Only one performance asset can be processed at the time | MetaHumanPerformance.h |
Variables
Public
| Name | Type | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|---|
| AnimationData | TArray64< struct FFrameAnimationData > | A 64 bit version of Animation Data array to support serialization of longer takes. | MetaHumanPerformance.h | |
| Audio | TObjectPtr< class USoundWave > | Audio of performance used with the Audio data input type | MetaHumanPerformance.h |
|
| AudioChannelIndex | uint32 | Specify the audio channel used to solve into animation | MetaHumanPerformance.h |
|
| AudioDrivenAnimationModels | FAudioDrivenAnimationModels | The models to be used by audio driven animation | MetaHumanPerformance.h |
|
| AudioDrivenAnimationOutputControls | EAudioDrivenAnimationOutputControls | MetaHumanPerformance.h |
|
|
| AudioDrivenAnimationSolveOverrides | FAudioDrivenAnimationSolveOverrides | Settings to change the behavior of the audio driven animation solve | MetaHumanPerformance.h |
|
| bAutoChooseHeadMovementReferenceFrame | uint8 | If set to true, automatically pick the most front - facing frame as the reference frame for control-rig head movement calculation, default to true. | MetaHumanPerformance.h |
|
| bDownmixChannels | bool | Downmix multi channel audio before solving into animation | MetaHumanPerformance.h |
|
| bGenerateBlinks | bool | Flag indicating if we should generate blinks | MetaHumanPerformance.h |
|
| bHeadStabilization | bool | Reduces noise in head position and orientation. | MetaHumanPerformance.h |
|
| bNeutralPoseCalibrationEnabled | bool | If set to true perform neutral pose calibration for mono solve, default to false. | MetaHumanPerformance.h |
|
| bRealtimeAudio | bool | Flag indicating if we should use realtime audio solve | MetaHumanPerformance.h |
|
| bShowFramesAsTheyAreProcessed | bool | Flag indicating if editor updates current frame to show the results as frames are processed | MetaHumanPerformance.h |
|
| bSkipDiagnostics | bool | Flag indicating whether processing diagnostics should be calculated during processing | MetaHumanPerformance.h |
|
| bSkipFiltering | bool | Flag indicating if filtering should be skipped | MetaHumanPerformance.h |
|
| bSkipPerVertexSolve | bool | Flag indicating if per-vertex solve (which is slow to process but gives slightly better animation results) should be skipped | MetaHumanPerformance.h |
|
| bSkipPreview | bool | Flag indicating if performance predictive solver preview should be skipped | MetaHumanPerformance.h |
|
| bSkipTongueSolve | bool | Flag indicating if tongue solving should be skipped | MetaHumanPerformance.h |
|
| Camera | FString | Name of camera (view) in the footage capture data calibration to use for display and processing | MetaHumanPerformance.h |
|
| CameraNames | TArray< TSharedPtr< FString > > | List of all RGB cameras (views) in the footage capture data | MetaHumanPerformance.h | |
| CaptureDataConfig | FString | Display name of the config to use with the capture data | MetaHumanPerformance.h |
|
| ContourTrackingResults | TArray64< FFrameTrackingContourData > | A 64 bit version of Contour Data array to support serialization of longer takes. | MetaHumanPerformance.h | |
| ControlRigClass | TSubclassOf< class UControlRig > | MetaHumanPerformance.h |
|
|
| DefaultSolver | TObjectPtr< class UMetaHumanFaceAnimationSolver > | Solver parameters for processing the footage | MetaHumanPerformance.h |
|
| DefaultTracker | TObjectPtr< class UMetaHumanFaceContourTrackerAsset > | Tracker parameters for processing the footage | MetaHumanPerformance.h |
|
| DepthMapDiagnosticResults | TArray< FDepthMapDiagnosticsResult > | Outputs. | MetaHumanPerformance.h | |
| EndFrameToProcess | uint32 | The frame to end processing with | MetaHumanPerformance.h |
|
| FocalLength | float | The estimated focal length of the footage | MetaHumanPerformance.h |
|
| FootageCaptureData | TObjectPtr< class UFootageCaptureData > | Real-world footage data with the performance | MetaHumanPerformance.h |
|
| HeadMovementMode | EPerformanceHeadMovementMode | Head movement type | MetaHumanPerformance.h |
|
| HeadMovementReferenceFrame | uint32 | Which frame to use as reference frame for head pose (if Auto Choose Head Movement Reference Frame is not selected), default to first processed frame. | MetaHumanPerformance.h |
|
| HeadMovementReferenceFrameCalculated | int32 | Head reference frame, calculated from the two properties above. | MetaHumanPerformance.h |
|
| Identity | TObjectPtr< class UMetaHumanIdentity > | A digital double of the person performing in the footage, captured in the MetaHuman Identity asset | MetaHumanPerformance.h |
|
| InputType | EDataInputType | Enum to indicate which data input type is being used for the performance | MetaHumanPerformance.h |
|
| MaximumScaleDifferenceFromIdentity | float | The maximum allowed percentage difference in estimated head scale between Identity and Performance. | MetaHumanPerformance.h |
|
| MaximumStereoBaselineDifferenceFromIdentity | float | The maximum allowed percentage difference in stereo baseline between Identity and Performance CaptureData camera calibrations. | MetaHumanPerformance.h |
|
| MinimumDepthMapFaceCoverage | float | The minimum percentage of the face region which should have valid depth-map pixels. | MetaHumanPerformance.h |
|
| MinimumDepthMapFaceWidth | float | The minimum required width of the face region on the depth-map in pixels. | MetaHumanPerformance.h |
|
| MonoSmoothingParams | TObjectPtr< UMetaHumanRealtimeSmoothingParams > | Smoothing parameters to use for mono video processing | MetaHumanPerformance.h |
|
| NeutralPoseCalibrationAlpha | double | Neutral pose calibration alpha parameter, defaults to 1. | MetaHumanPerformance.h |
|
| NeutralPoseCalibrationCurves | TArray< FName > | Set of curve names to apply neutral pose calibration to. | MetaHumanPerformance.h |
|
| NeutralPoseCalibrationFrame | uint32 | Which frame to use as the neutral pose calibration for mono solve (if Enable Neutral Pose Calibration is selected), default to first processed frame. | MetaHumanPerformance.h |
|
| OnProcessingFinishedDynamic | FOnProcessingFinishedDynamic | Dynamic delegate called when the pipeline finishes running. | MetaHumanPerformance.h |
|
| ProcessingExcludedFrames | TArray< FFrameRange > | Frames that the processing has identified as producing bad results and should not be exported | MetaHumanPerformance.h |
|
| ScaleEstimate | float | MetaHumanPerformance.h | ||
| SolveType | ESolveType | Enum to indicate which type of solve to perform | MetaHumanPerformance.h |
|
| StartFrameToProcess | uint32 | The frame to start processing from | MetaHumanPerformance.h |
|
| TimecodeAlignment | ETimecodeAlignment | Timecode alignment type | MetaHumanPerformance.h |
|
| UserExcludedFrames | TArray< FFrameRange > | Frames that the user has identified which are to be excluded from the processing, eg part of the footage where the face goes out of frame | MetaHumanPerformance.h |
|
| ViewportSettings | TObjectPtr< class UMetaHumanPerformanceViewportSettings > | Stores the viewport settings used in the Performance asset editor | MetaHumanPerformance.h | |
| VisualizationMesh | TObjectPtr< class USkeletalMesh > | Set a different Skeletal Mesh (e.g. MetaHuman head) for visualizing the final animation | MetaHumanPerformance.h |
|
Protected
| Name | Type | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|---|
| AnimationResultsPinName | FString | MetaHumanPerformance.h | ||
| AudioDrivenAnimationViewportTransform | const FTransform | Rotation and push back needed so things appear correctly in the viewport. | MetaHumanPerformance.h | |
| bBlockingProcessing | bool | MetaHumanPerformance.h | ||
| bEstimateFocalLengthOK | bool | MetaHumanPerformance.h | ||
| bIsScriptedProcessing | bool | MetaHumanPerformance.h | ||
| bMetaHumanAuthoringObjectsPresent | bool | MetaHumanPerformance.h | ||
| DepthMapDiagnosticsResultsPinName | FString | MetaHumanPerformance.h | ||
| EstimateFocalLengthErrorMessage | FString | MetaHumanPerformance.h | ||
| MediaFrameRanges | TMap< TWeakObjectPtr< UObject >, TRange< FFrameNumber > > | MetaHumanPerformance.h | ||
| OnControlRigClassChangedDelegate | FOnControlRigClassChanged | MetaHumanPerformance.h | ||
| OnDataInputTypeChangedDelegate | FOnDataInputTypeChanged | MetaHumanPerformance.h | ||
| OnExcludedFramesChangedDelegate | FOnExcludedFramesChanged | MetaHumanPerformance.h | ||
| OnFrameProcessedDelegate | FOnFrameProcessed | MetaHumanPerformance.h | ||
| OnFrameRangeChangedDelegate | FOnFrameRangeChanged | MetaHumanPerformance.h | ||
| OnGetCurrentFrameDelegate | FFrameRangeArrayBuilder::FOnGetCurrentFrame | MetaHumanPerformance.h | ||
| OnHeadMovementModeChangedDelegate | FOnHeadMovementModeChanged | MetaHumanPerformance.h | ||
| OnHeadMovementReferenceFrameChangedDelegate | FOnHeadMovementReferenceFrameChanged | MetaHumanPerformance.h | ||
| OnIdentityChangedDelegate | FOnIdentityChanged | MetaHumanPerformance.h | ||
| OnNeutralPoseCalibrationChangedDelegate | FOnNeutralPoseCalibrationChanged | MetaHumanPerformance.h | ||
| OnProcessingFinishedDelegate | FOnProcessingFinished | MetaHumanPerformance.h | ||
| OnRealtimeAudioChangedDelegate | FOnRealtimeAudioChanged | MetaHumanPerformance.h | ||
| OnSourceDataChangedDelegate | FOnSourceDataChanged | MetaHumanPerformance.h | ||
| OnStage1ProcessingFinishedDelegate | FOnStage1ProcessingFinished | MetaHumanPerformance.h | ||
| OnVisualizeMeshChangedDelegate | FOnVisualizeMeshChanged | MetaHumanPerformance.h | ||
| OverrideVisualizationMesh_DEPCRECATED | TObjectPtr< class USkeletalMesh > | MetaHumanPerformance.h | ||
| PipelineExcludedFrames | TArray< FFrameRange > | MetaHumanPerformance.h | ||
| PipelineFrameRanges | TArray< FFrameRange > | MetaHumanPerformance.h | ||
| PipelineFrameRangesIndex | int32 | MetaHumanPerformance.h | ||
| Pipelines | TArray< TSharedPtr< UE::MetaHuman::Pipeline::FPipeline > > | MetaHumanPerformance.h | ||
| PipelineStage | int32 | MetaHumanPerformance.h | ||
| PipelineStageStartTime | double | MetaHumanPerformance.h | ||
| PreviousTimecodeAlignment | ETimecodeAlignment | MetaHumanPerformance.h | ||
| ProcessingLimitFrameRange | TRange< FFrameNumber > | MetaHumanPerformance.h | ||
| ProcessingStartTime | double | MetaHumanPerformance.h | ||
| RateMatchingExcludedFrames | TArray< FFrameRange > | MetaHumanPerformance.h | ||
| RealtimeMonoSolver | TSharedPtr< UE::MetaHuman::Pipeline::FHyprsenseRealtimeNode > | MetaHumanPerformance.h | ||
| RealtimeSpeechToAnimSolver | TSharedPtr< UE::MetaHuman::Pipeline::FRealtimeSpeechToAnimNode > | MetaHumanPerformance.h | ||
| ScaleDiagnosticsResultsPinName | FString | MetaHumanPerformance.h | ||
| SolverConfigData | FString | MetaHumanPerformance.h | ||
| SolverDefinitionsData | FString | MetaHumanPerformance.h | ||
| SolverHierarchicalDefinitionsData | FString | MetaHumanPerformance.h | ||
| SolverTemplateData | FString | MetaHumanPerformance.h | ||
| SpeechToAnimSolver | TSharedPtr< UE::MetaHuman::Pipeline::FSpeechToAnimNode > | MetaHumanPerformance.h | ||
| TongueSolver | TSharedPtr< UE::MetaHuman::Pipeline::FTongueTrackerNode > | MetaHumanPerformance.h | ||
| TrackingResultsPinName | FString | MetaHumanPerformance.h |
Functions
Public
| Name | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|
FTransform AudioDrivenHeadPoseTransform
(
const FTransform& InHeadBonePose |
MetaHumanPerformance.h | ||
FTransform AudioDrivenHeadPoseTransformInverse
(
const FTransform& InRootBonePose |
MetaHumanPerformance.h | ||
FTransform CalculateReferenceFramePose() |
Calculate the head pose from the AnimationData, either using specified reference frame, or auto-selecting the best one | MetaHumanPerformance.h | |
void CancelPipeline() |
MetaHumanPerformance.h |
|
|
bool CanExportAnimation() |
MetaHumanPerformance.h |
|
|
bool CanProcess() |
MetaHumanPerformance.h |
|
|
bool ContainsAnimationData() |
Returns true if there is at least one animation frame with valid data, false otherwise | MetaHumanPerformance.h |
|
bool DepthCameraConsistentWithRGBCameraOrDiagnosticsNotEnabled() |
Returns true if the depth camera location is consistent with the RGB camera location, or diagnostics are not enabled | MetaHumanPerformance.h | |
bool DiagnosticsIndicatesProcessingIssue
(
FText& OutDiagnosticsWarningMessage |
MetaHumanPerformance.h |
|
|
bool EstimateFocalLength
(
FString& OutErrorMessage |
Estimate the focal length of the footage | MetaHumanPerformance.h | |
void ExportAnimation
(
EPerformanceExportRange InExportRange |
(DEPRECATED: use UMetaHumanPerformanceExportUtils::ExportAnimation instead) Export an animation sequence targeting the face skeleton. | MetaHumanPerformance.h |
|
TSet< FString > GetAnimationCurveNames() |
Returns a list of animation curves used by this Performance | MetaHumanPerformance.h | |
TArray< struct FFrameAnimationData > GetAnimationData
(
int32 InStartFrameNumber, |
Returns animation data (frame numbers are animation frame index not sequencer frame numbers) Caller is responsible to ensure data will fit into 32bit TArray | MetaHumanPerformance.h |
|
TObjectPtr< class USoundWave > GetAudioForProcessing() |
Getter for audio to use for processing. Either audio in FootageCaptureData or overridden by Audio | MetaHumanPerformance.h | |
FTimecode GetAudioMediaTimecode() |
Getter for timecode from audio media | MetaHumanPerformance.h | |
FFrameRate GetAudioMediaTimecodeRate() |
Getter for timecode rate from audio media | MetaHumanPerformance.h | |
FText GetCannotProcessTooltipText() |
Returns tooltip text with a reason why processing cant be started | MetaHumanPerformance.h | |
EFrameRangeType GetExcludedFrame
(
int32 InFrameNumber |
MetaHumanPerformance.h | ||
TRange< FFrameNumber > GetExportFrameRange
(
EPerformanceExportRange InExportRange |
MetaHumanPerformance.h | ||
FTransform GetFirstValidAnimationPose() |
Returns the first valid pose in the AnimationData | MetaHumanPerformance.h | |
FFrameRate GetFrameRate() |
MetaHumanPerformance.h | ||
FString GetHashedPerformanceAssetID() |
MetaHumanPerformance.h | ||
const TMap< TWeakObjectPtr< UObject >, TRange< FFrameNumber > > & GetMediaFrameRanges() |
MetaHumanPerformance.h | ||
FFrameNumber GetMediaStartFrame() |
MetaHumanPerformance.h | ||
int32 GetNumberOfProcessedFrames() |
MetaHumanPerformance.h |
|
|
int32 GetPipelineStage() |
MetaHumanPerformance.h | ||
const TRange< FFrameNumber > & GetProcessingLimitFrameRange() |
MetaHumanPerformance.h | ||
| Returns the effective Skeletal Mesh used for visualization. | MetaHumanPerformance.h | ||
bool HasValidAnimationPose() |
Returns if any frame has valid pose in the AnimationData | MetaHumanPerformance.h | |
bool IsProcessing() |
MetaHumanPerformance.h |
|
|
FOnControlRigClassChanged & OnControlRigClassChanged() |
MetaHumanPerformance.h | ||
FOnDataInputTypeChanged & OnDataInputTypeChanged() |
MetaHumanPerformance.h | ||
FOnExcludedFramesChanged & OnExcludedFramesChanged() |
MetaHumanPerformance.h | ||
FOnFrameProcessed & OnFrameProcessed() |
MetaHumanPerformance.h | ||
FOnFrameRangeChanged & OnFrameRangeChanged() |
MetaHumanPerformance.h | ||
FFrameRangeArrayBuilder::FOnGetCurrentFrame & OnGetCurrentFrame() |
MetaHumanPerformance.h | ||
FOnHeadMovementModeChanged & OnHeadMovementModeChanged() |
MetaHumanPerformance.h | ||
FOnHeadMovementReferenceFrameChanged & OnHeadMovementReferenceFrameChanged() |
MetaHumanPerformance.h | ||
FOnIdentityChanged & OnIdentityChanged() |
MetaHumanPerformance.h | ||
FOnNeutralPoseCalibrationChanged & OnNeutralPoseCalibrationChanged() |
MetaHumanPerformance.h | ||
FOnProcessingFinished & OnProcessingFinished() |
MetaHumanPerformance.h | ||
FOnRealtimeAudioChanged & OnRealtimeAudioChanged() |
MetaHumanPerformance.h | ||
FOnSourceDataChanged & OnSourceDataChanged() |
MetaHumanPerformance.h | ||
FOnStage1ProcessingFinished & OnStage1ProcessingFinished() |
MetaHumanPerformance.h | ||
FOnVisualizeMeshChanged & OnVisualizeMeshChanged() |
MetaHumanPerformance.h | ||
void SetBlockingProcessing
(
bool bInBlockingProcessing |
MetaHumanPerformance.h |
|
|
EStartPipelineErrorType StartPipeline
(
bool bInIsScriptedProcessing |
Export options. | MetaHumanPerformance.h |
|
Overridden from UObject
| Name | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|
virtual void BeginDestroy() |
MetaHumanPerformance.h | ||
virtual bool CanEditChange
(
const FProperty* InProperty |
MetaHumanPerformance.h | ||
virtual FPrimaryAssetId GetPrimaryAssetId() |
MetaHumanPerformance.h | ||
virtual void PostEditChangeProperty
(
FPropertyChangedEvent& InPropertyChangedEvent |
MetaHumanPerformance.h | ||
virtual void PostEditUndo() |
MetaHumanPerformance.h | ||
virtual void PostInitProperties() |
MetaHumanPerformance.h | ||
virtual void PostLoad() |
MetaHumanPerformance.h | ||
virtual void PreEditChange
(
FEditPropertyChain& InPropertyAboutToChange |
MetaHumanPerformance.h | ||
virtual void Serialize
(
FArchive& Ar |
MetaHumanPerformance.h |
Static
| Name | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|
static FVector GetSkelMeshReferenceBoneLocation
(
USkeletalMeshComponent* InSkelMeshComponent, |
Returns a bone position in the reference skeleton - used to account for variations in the heights of different MetaHumans | MetaHumanPerformance.h |
Deprecated Variables
| Name | Type | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|---|
| ControlRig_DEPRECATED | TObjectPtr< class UControlRigBlueprint > | Control Rig used to drive the animation | MetaHumanPerformance.h |