It is possible to animate a MetaHuman character in real time from a wide range of capture devices using Live Link. In the case of a mobile device, the Live Link Face application is also required. You can learn more about capturing your facial performance by reading the Facial Performance Capture Guidelines.
Prerequisites
To generate animation in real time, you need:
An Unreal Engine 5.6 (or later) project.
The MetaHuman Live Link plugin enabled.
A connected audio, mono video or mobile device running the Live Link Face app.
By default the Live Link window may contain an ARKit source. This is not required to animate your MetaHuman in real time and we recommend deleting it by clicking the trash button.
Generating Realtime Animation
The workflow to generate realtime animation consists of the following steps:
Open the Live Link window by navigating to Window > Virtual Production > Live Link.
Create a Live Link subject using the Live Link source based on the device that is being used for capture.
Connect the subject to the MetaHuman character, either for preview in MetaHuman Creator or to animate in a level.
Running in Live Link Hub
The MetaHuman Live Link sources can be created either in the Live Link manager tab of Unreal Engine as described above, or in Live Link Hub.
It is only recommended to run in Live Link Hub when Live Link Hub will be running on a separate machine from Unreal Engine.
The MetaHuman Realtime Video and Audio Live Link sources can be computationally expensive and utilize the GPU. If these are run in Live Link Hub, it can lead to resource contention with Unreal Engine itself which may also be making heavy use of the GPU. Unreal Engine and Live Link Hub are two independent processes and the priority, scheduling and resource allocation of these two tasks will be down to the OS and/or graphics drivers. This can lead to one process having the majority share of resources at the cost of the other (which we have found to typically be the task which has mouse focus). A low processing rate due to resource contention will reduce the fidelity of the animation that is generated.
To simplify matters, we recommend running the MetaHuman Live Link sources in Unreal Engine's Live Link panel when possible. With all GPU work being carried out in the single process, Unreal Engine is better able to balance the rendering load with the real-time video and audio processing load.
If you do wish to use Live Link Hub, we strongly recommend running it on a separate machine to Unreal Engine itself.
If that is not possible, then we suggest fixing an upper frame rate in Unreal Engine. Without an upper frame rate limit set, Unreal Engine renders the scene as fast as possible, which can easily exceed 100fps for simple scenes. By placing an upper frame rate limit (using the cvar t.maxfps), the GPU load from Unreal Engine is reduced, leaving more resources available to Live Link Hub. This results in a faster rate of animation generation and a better balance between this and rendering.
It can also be beneficial to limit the speed of the game thread in Live Link Hub to 60fps by using the same cvar and t.maxfps described above. This can help to provide more resources to the real-time processing.