To generate real-time animation from a video device, media asset or stream source, it is necessary to create a MetaHuman Video source in Live Link, and then configure the new subject.
While this page refers to a webcam, any connected USB Video Class (UVC) compatible video capture device supported by Windows or macOS can be used. Windows 11 is required to use a software virtual camera as a webcam on Windows. On macOS, virtual camera support depends on the application and installed virtual camera software. This feature is only supported on Windows and macOS.
Create a MetaHuman Video Live Link Source
Click Add Source and select the MetaHuman (Video) option.
Select the newly created source to show the Live Link source settings in the right hand panel.
From the Video Device Type dropdown, select the type of video source you want to use:
Capture Device (default): A physically connected camera of video capture device (webcam, video card etc)
Media Source: A UmediaSource asset in your project.
Media Bundle: a UMediaBundle asset in your project.
Media Profile: a UMediaProfile asset in your project.
When running in Live Link Hub, only Capture Device and Media Profile types are available.
Select an available device from the Video Device field. The picker changes depending on the device type:
Capture Device: A dropdown listing all connected capture devices (webcams, capture cards). Click the Refresh button to re-enumerate.
Media Source / Media Bundle / Media Profile: An asset picket showing all matching assets in your project.
Select the desired Video Format. This value dictates the frame rate and resolution at which the video will be captured. It is recommended to select the format with the highest frame rate your camera can support, and then choose the highest resolution supported at that frame rate.
This option is only available when Video Device Type is set to Capture Device. For Media Source, Media Bundle, and Media Profile types, the format is defined by the asset and does not need manual selection.
Optionally specify a Subject Name. This is used to configure the MetaHuman for Live Link animation input. If not specified, the Video Device name will be used.
(Optional) Configure the Advanced settings:
Setting Description Video Track
The active video track to use for devices that support more than one video track. The most suitable video track is automatically selected by default. (Only available for Capture Device type; disabled for Media Source, Media Bundle, and Media Profile types)
Filter Format List
When enabled, the Video Format list is filtered to only show formats which are suitable for real time animation processing. This option removes formats that are less than 500x500 resolution or less than 24fps.
Start Timeout
Timeout for waiting for the webcam to open once Connect is pressed. If the webcam does not respond within this time an error occurs and real time animation is not available.
Format Wait Time
The time to wait for the webcam to acknowledge the selection of which video format to use.
Sample Timeout
The time to wait for the first video frame to arrive from the webcam.
Models
Manually select the models used for real-time facial animation processing. The options include:
Face Detector model: The user can choose between the standard Face Detector model, or a Small Face Detector model, which works better when the face occupies a small fraction of the image.
Face Head Pose Tracker model: A single HeadPoseTracker model is available.
Face Solver model: The user can choose between the standard Generic Rig Solver model, which works well with all types of footage, the Stereo HMC Rig Solver model, which works better on (mono) footage from a greyscale Visible Light or IR Stereo HMC capture device, or finally, the Small Face Solver mode, which works well when the face occupies a small fraction of the image.
Click Connect. This will start the camera video stream and create a subject with the chosen subject name.
When successfully connected, the new subject will appear in the list of available subjects with a green tick icon to show that it is operating normally. An amber warning triangle could be caused by the face not being visible to the camera.
Using a Media Assets as the Video Source
Any Media Source, Media Bundle or Media Profile assets in the project can be selected as a Video Device and can act as a video source. They are supported in the same way as a webcam.
Media Source: A UMediaSource asset wrapping any media format supported by the Media Framework (video files, network streams etc).
Media Bundle: A UMediaBundle asset that combines a media source with a player and texture. Useful for pre-configured setups like streaming a movie file, playing an image sequence or reading from a capture card.
Media Profile: a UMediaProfile asset defining multiple media sources for multi-source studio configurations.
When using any of these media asset types, the Video Track and Video Format fields are disabled since these are defined by the asset itself.
Configure the MetaHuman Video Live Link Subject
Selecting the new subject within the list will show the subject settings where you can view the camera feed in the Monitor Image section.
Alternative options for the Monitor Image are:
None: No image.
Input: The raw video.
Trackers: The video with tracking markers overlaid. This can be useful to analyze the stability of the animation solve.
Monitoring the video feed takes some computational load. If performance is paramount, especially at high webcam frames rates or heavily loaded scene, you may wish to use this feature sparingly.
The default value for this control can be set in the Project Preferences.
The Information section provides details about the animation solve process:
| Information | Description |
|---|---|
State | An at-a-glace summary of the state of the processing. OK indicates a good state, while other messages, like “No face detected,” are hopefully self-explanatory and indicate there is a problem. The state is also reflected in an LED-like indicator with green for good, yellow for less than ideal conditions but animation can still be generated (for example, subject too far from camera, low-lighting); amber for a transitory failure to generate animation (for example, no face detected); and red for an unrecoverable failure (for example, webcam unplugged). |
Resolution | The input video resolution. |
Dropping | Whether video frames are being dropped because they cannot be processed fast enough. Ideally this should read “No.” If it reads “Yes” then the webcam is running faster than the processing and video frames are being dropped in order for the processing not to lag behind the video. The number of frames being dropped in a 2 second period is shown. |
FPS | The processing frame rate. This can be less than or equal to the webcam frame rate if frames are being dropped. When the processing rate is not equal to the webcam frame rate, both rates are shown. |
Timecode | The timecode of the frame being processed. The timecode can come from the device itself or, if it does not provide a timecode, from the system clock. |
Once configured, the new subject is ready to use to animate your MetaHuman character. For further configuration options, refer to Configuring a MetaHuman Video Subject.
Performance Troubleshooting
If you experience slow animation rates when using a webcam, open the Live Link subject properties and look at the Dropping field. If this is set to "Yes" then the webcam is running faster than the processing and video frames are being dropped in order for the processing to not lag behind the video. In order to improve the processing rate, you need to reduce the load on your machine. For example, lower the scalability settings, simplify your scene, or set a fixed frame rate for your scene in the project settings.
One important case to mention here is if you are running the real-time animation in Live Link Hub, contention between the Live Link Hub and UE processes can result in low animation rates. See the Running in Live Link Hub section for more details.
If the Dropping field is "No," the animation is webcam frame rate limited. The true frame rate of the webcam can be lower than you expect. While you can request a particular frame rate from the webcam, this is an "up to" figure, but the true frame rate you actually get relies on many factors. Make sure you have the webcam plugged directly into the computer as using an extender or hub can impact speed. Similarly, ensure you are using the highest speed USB cable and port supported by your webcam and computer to maximize bandwidth.
Lighting can impact frame rate. In low lighting conditions the webcam may have to use a longer exposure time to achieve a well-lit image and that can impact frame rate. Make sure you are well lit. Ring lights are ideal. (Low lighting may have other negative effects besides low frame rate such as the camera having to use a high gain, which increases the amount of noise in the image and can lead to jitter in the animation).
