From Video Footage | MetaHuman Documentation

This page outlines the Mesh to MetaHuman workflow for creating a MetaHuman Identity from video footage. The footage must have been captured using a compatible device, and ingested using Capture Manager in Live Link Hub.

As of Unreal Engine 5.6, the Mesh to MetaHuman workflow will no longer create a new character in the MetaHuman Creator web application. After creating the identity, use the Conform from Identity option in the MetaHuman Character asset editor to create the character directly in MetaHuman Creator in Unreal Engine.

Workflow

The Mesh to MetaHuman from video footage workflow consists of the following steps:

Import one or more takes that meet the performance capture requirements to Unreal Engine.
Create and populate a MetaHuman Identity asset.
Promote and track the required frames to run the Identity Solve.
Run the Identity Solve.
Submit the Template Mesh to the MetaHuman Backend.

These steps are covered in detail below.

Import Takes into Unreal Engine

Follow the workflow outlined on the Ingest Footage Using Capture Manager in Live Link Hub page.

After completing this workflow, you should have one or more takes that are ready to be imported into the MetaHuman Identity asset.

Create and Populate a MetaHuman Identity Asset

Next, you will create and populate a MetaHuman Identity Asset. This Asset holds the MetaHuman's face mesh (template mesh) and pose information.

Follow the steps below:

Create a new MetaHuman Identity Asset by right-clicking in the Content Browser. From the context menu, select MetaHuman > MetaHuman Identity.
If you create a MetaHuman from this Asset later, the filename of the Identity Asset you create will be your MetaHuman’s name in MetaHuman Creator.
Double-click the MetaHuman Identity Asset you created to open it in the Identity Asset Editor. Initially, this Asset is empty. You will populate it with data in the next steps.
In the Main Toolbar, click Create Components, then select From Footage. Search for and select a take to create the footage from.
After you complete this step, the Components panel on the left side of the Metahuman Identity Asset Editor will be populated with Face and Poses.

Promote and Track Required Frames

If using footage captured on an iPhone or iPad using a TrueDepth Camera, promote frames for the neutral pose, left view, right view, and teeth pose. For footage captured using a stereo camera pair, promote frames for the neutral and teeth poses.

To promote a frame, follow these steps:

In the Components panel, under Poses, select the Neutral Pose Component (1).
Use the video playback controls (2) to navigate to the frame you want to promote. You can also use the video timeline for precise seeking.
Click the plus button (3) to promote the frame. The first promoted frame is automatically treated as the front view.
Click the Free Roaming Camera Mode button (4) to unlock playback controls so you can promote another frame.

Right-click a frame to access a context menu with options to rename and demote (remove) a frame, or toggle which frame is treated as the front view.

Run the Identity Solve

Still in the Identity Asset Editor, from the Main Toolbar, click the MetaHuman Identity Solve button. This button will only be enabled once at least one Promoted Frame exists in the timeline, and will produce good results only when the markers are active and well-tracked.

Identity Solve fits the Template Mesh vertices to the volume of the Neutral Pose mesh you tracked in previous steps. This part of the process happens in the cloud.

After the Identity Solve completes, you can toggle between your original mesh and the Template Mesh in the Viewport by clicking the A / B tabs in the Viewport Toolbar. The A and B buttons activate two separate frame buffers for the same Viewport camera, and promoting a frame creates a snapshot of that camera. You can configure Viewport settings, such as lighting, individually for each of these two frame buffers.

A/B comparison between original footage and template mesh in the Viewport. Also visible are the Viewport Toolbar (1) that contains camera and preview controls, as well as individual settings for each frame buffer (2). You can choose different frames (3) to compare the Template Mesh output with.

Submit the Template Mesh for Auto-Rigging

You can now submit the Template Mesh to the MetaHuman backend. In the Main Toolbar, click the Auto-Rig MetaHuman Identity button.

This creates an auto-rigged Skeletal Mesh with embedded MetaHuman DNA. The mesh is automatically downloaded and added to your Unreal Engine project in the same folder as the MetaHuman Identity Asset and has the same name as the Identity Asset, with the prefix SK_.

Unreal Engine will show a confirmation dialog when the process has finished. You will then be able to see the new Skeletal Mesh in the Content Browser.

Next Up

From Mesh

Create a MetaHuman Identity from mesh data.