To generate realtime animation from a mono video device, it is necessary to create a MetaHuman Video source in Live Link, and then configure the new subject.
This feature is only supported on Windows.
Create a MetaHuman Video Live Link Source
Click Add Source and select the MetaHuman (Video) option.
Select the newly created source to show the Live Link source settings in the right hand panel.
Select an available Video Device from the Create section.
Select the desired Video Format. This value dictates the frame rate and resolution at which video will be captured. We recommend selecting the format with the highest frame rate your camera can support, then, secondly, the highest resolution supported at that frame rate.
Optionally specify a Subject Name. This is used to configure the MetaHuman for Live Link animation input. If not specified, the Video Device name will be used.
(Optional) Configure the Advanced settings:
Setting Description Video Track
The active video track to use for devices that support more than one video track. The most suitable video track is automatically selected by default.
Filter Format List
When enabled, the Video Format list is filtered to only show formats which are suitable for real time animation processing. This option removes formats that are less than 500x500 resolution or less than 24fps.
Start Timeout
Timeout for waiting for the webcam to open once Connect is pressed. If the webcam does not respond within this time an error occurs and real time animation is not available.
Format Wait Time
The time to wait for the webcam to acknowledge the selection of which video format to use.
Sample Timeout
The time to wait for the first video frame to arrive from the webcam.
Click Connect. This will start the camera video stream and create a subject with the chosen subject name.
When successfully connected, the new subject will appear in the list of available subjects with a green tick icon to show that it is operating normally. An amber warning triangle could be caused by the face not being visible to the camera.
Using a Media Bundle as the Video Device
Any Media Bundle assets in the project will be listed as a Video Device and can act as a video source, either through streaming a movie file, image sequence or reading from a capture card. They are supported in the same way as a webcam.
Configure the MetaHuman Video Live Link Subject
Selecting the new subject within the list will show the subject settings where you can view the camera feed in the Monitor Image section.
Alternative options for the Monitor Image are:
None: No image.
Input: The raw video.
Trackers: The video with tracking markers overlaid. This can be useful to analyze the stability of the animation solve.
Monitoring the video feed takes some computational load. If performance is paramount, especially at high webcam frames rates or heavily loaded scene, you may wish to use this feature sparingly.
The default value for this control can be set in the Project Preferences.
The Information section provides details about the animation solve process:
| Information | Description |
|---|---|
State | An at-a-glace summary of the state of the processing. OK indicates a good state, while other messages, like “No face detected,” are hopefully self-explanatory and indicate there is a problem. The state is also reflected in an LED-like indicator with green for good, yellow for less than ideal conditions but animation can still be generated (for example, subject too far from camera, low-lighting); amber for a transitory failure to generate animation (for example, no face detected); and red for an unrecoverable failure (for example, webcam unplugged). |
Resolution | The input video resolution. |
Dropping | Whether video frames are being dropped because they cannot be processed fast enough. Ideally this should read “No.” If it reads “Yes” then the webcam is running faster than the processing and video frames are being dropped in order for the processing not to lag behind the video. The number of frames being dropped in a 2 second period is shown. |
FPS | The processing frame rate. This can be less than or equal to the webcam frame rate if frames are being dropped. When the processing rate is not equal to the webcam frame rate, both rates are shown. |
Timecode | The timecode of the frame being processed. The timecode can come from the device itself or, if it does not provide a timecode, from the system clock. |
Once configured, the new subject is ready to use to animate your MetaHuman character. For further configuration options, refer to Configuring a MetaHuman Video Subject.
Performance Troubleshooting
If you experience slow animation rates when using a webcam, open the Live Link subject properties and look at the Dropping field. If this is set to "Yes" then the webcam is running faster than the processing and video frames are being dropped in order for the processing to not lag behind the video. In order to improve the processing rate, you need to reduce the load on your machine. For example, lower the scalability settings, simplify your scene, or set a fixed frame rate for your scene in the project settings.
One important case to mention here is if you are running the real-time animation in Live Link Hub, contention between the Live Link Hub and UE processes can result in low animation rates. See the Running in Live Link Hub section for more details.
If the Dropping field is "No," the animation is webcam frame rate limited. The true frame rate of the webcam can be lower than you expect. While you can request a particular frame rate from the webcam, this is an "up to" figure, but the true frame rate you actually get relies on many factors. Make sure you have the webcam plugged directly into the computer as using an extender or hub can impact speed. Similarly, ensure you are using the highest speed USB cable and port supported by your webcam and computer to maximize bandwidth.
Lighting can impact frame rate. In low lighting conditions the webcam may have to use a longer exposure time to achieve a well-lit image and that can impact frame rate. Make sure you are well lit. Ring lights are ideal. (Low lighting may have other negative effects besides low frame rate such as the camera having to use a high gain, which increases the amount of noise in the image and can lead to jitter in the animation).