Calibration is a prerequisite for generating depth data, and therefore necessary to solve animation in MetaHuman Animator from a stereo camera pair. It can be generated directly in Unreal Engine from footage of a calibration (or checker) board. Refer to the Calibration Capture Guidelines page for information on how best to capture this footage.
Prerequisites
Ingest the calibration board footage using Live Link Hub. This will create a Capture Data asset.
Using the Calibration Generator Tool
Right-click on the Capture Data asset in the Content Browser and select the Generate Calibration option from the menu.
This will open the Calibration Generator tool, showing the images from each camera view in the calibration take.
The Calibration Generator Tool consists of the following components:
Config > : Can be saved and re-opened for later use to enable repeatable calibration across multiple takes.
Run Auto Frame Selection
Run Calibration
Project Settings
Images of capture from both cameras
View settings for each camera capture.
Reset View will not remove an Area of Interest that has been selected. Using this function will reset the zoom/panning.
Frame selection slider and control buttons
+ Add frame: Clicking on the + will add a visual representation of the points coverage that are detected on the selected frame.
- Remove frame.
Pressing the keyboard spacebar will add or remove (+/-) the current frame.
< > Move between frames.
Pressing A or D on the keyboard will move by one between frames.
Pressing Shift-A or Shift-D will move by the framerate.
Reset: Resets the entire calibration.
Config tab
Options tab
| Property | Description |
|---|---|
| Config | |
Board Pattern Width | The number of squares on the width of the board. |
Board Pattern Height | The number of squares on the height of the board. |
Board Square Size | The edge size of each square in cm. |
| Options | |
Asset Name | The name of the calibration being used. |
Package Path | The location within the project in which to store the created assets. |
Auto Save Assets | Whether to automatically save the generated calibration assets. |
Sharpness Threshold | The allowed blurriness (in pixels) of the frame that will be used. Any frames with an estimated blurriness above this threshold will be ignored. |
Area of Interests for Cameras | This option plays a role in the Automatic Frame Selection. The process will prioritize frames in which the checkerboard covers most of the specified area, or all of it. If the area of interest isn't specified, the process will prioritize frames where the checkerboard covers the central part of the frame. |
Automatic Frame Selection
This process will go through every nth frame (as determined by the Automatic Frame Selection Sample Rate in the Project Settings) and select the frames with the most pose diversity to capture as many points as possible. The variety of the checkerboard positions is therefore important for the calibration process.
Pose diversity is based on the coverage of the checkerboard in that frame and the variety of positions of the checkerboard in the footage. Both of these define a score for the frame, and only the frames with the highest score will be part of the process's result. It will discard frames for which the checkerboard position hasn’t changed and so only choose frames with a unique checkerboard position.
Running this process on its own may exclude some areas that are of importance to your calibration. If the calibration didn’t include these areas, you can select an area of interest and rerun the calibration.
Click Run Auto Frame Selection.
A warning will be shown if the automatic process is unable to detect sufficient frames. The threshold can be configured in the project settings.
Additional frames can be selected manually if required.
Selecting an Area of Interest
The default area of interest assumes the checkerboard is predominantly in the middle of the frame, which has been found to give good results with most stereo HMC hardware.
However, you can optionally select an area of interest if the checkerboard is not centrally framed or the lens choice means it occupies a smaller section of the frame. This will prioritize coverage within the new area of interest, reducing the influence of other areas in the image.
If you want to limit or expand the coverage area, you can achieve this with the following steps.
Click on the image settings button and click Select Area of Interest.
Using your cursor, click and drag to create a selection area box.Repeat for each image.
Reset View will not remove an Area of Interest that has been manually selected. Either close and reopen the Calibration Generator tool, or zoom out and select an area of interest that incorporates the entire frame.
Manual Frame Selection
Using the slider and frame selection button, add as many frames to capture all the necessary points.
For optimal calibration, select frames which are not blurry, have the checkerboard fully in frame and in a variety of positions, then run the calibration to view the results. If the coverage is not sufficient, add or change frames and run the calibration again.
Repeat this until you obtain the most "green" coverage.
Running Calibration
As frames are selected (either automatically or manually), the view will update with a visual representation of the entire coverage area. The colours are defined based on the required number of points for a specific block in the frame, while the number represents the percentage of how much the required coverage is satisfied.
Once you are happy with the calibration coverage, click the Run Calibration button to generate the calibration assets. When finished, a popup will be displayed with a root mean squared (RMS) reprojection error rate, giving an indication as to the quality of the calibration.
The RMS reprojection error ranges are given below. These should be taken as guidance only; as a mathematical measurement of the process, a value reported as ‘good’ or even ‘excellent’ may still result in poor depth or animation data.
0.0 px < RMS < 0.3 px : Excellent
0.3 px < RMS < 0.7 px : Good
0.7 px < RMS < 1.0 px : Acceptable
1.0 px < RMS < 1.5 px : Borderline
1.5 px < RMS : Bad
Once complete, a Camera Calibration asset referencing a LensFile asset for each camera will have been created in the target folder. The LensFile asset is part of the Unreal Engine Virtual Production toolkit.