The Calibration Diagnostics tool provides a way of checking the calibration on the performance footage. For example, if you have captured calibration at the start and end of a capture session, this tool can be used to evaluate which of the two calibrations should be used with a performance take.
This is an experimental feature. To use it, you must enable the MetaHuman Animator Calibration Diagnostics Plugin in your project.
To open the tool, right-click on the Capture Data asset for a performance take and select the Calibration Diagnostics option from the menu.
In the Options tab, select the camera calibration asset you intend to use. Once selected, click the Detect Features button to run the diagnostics for the currently selected frame.
Select another frame and click Detect Features again to continue verifying that the error rate is within a good range across the take.
The Image Viewer tab provides a few diagnostic visualizations which can be toggled using the Viewer settings:
With Detected Points enabled, features and their reprojections are displayed with a line connecting the two points. These errors give visual representation of how good the calibration is for the current footage, measuring the distance between the camera point and the reprojection point. The line is colored based on the error severity.
Green represents a small error (distance) between two points.
Red represents a big error between the points.
With Per Block Errors enabled the image is split into blocks, and every block is colored depending on the Root Mean Squared (RMS) reprojection error. Clicking on one of the blocks enables you to see other reprojection error types (mean, median) and a count of how many points are detected.
Selecting an Area of Interest
By default, features will be detected across the entire frame. This may mean that background elements are inadvertently detected, influencing the overall result. If this is the case, an area of interest that more specifically frames the face can be defined to discard these values.
If you want to limit or expand the coverage area, you can achieve this with the following steps.
Click on the arrow button next to the viewer settings
Using your cursor, click and drag to create a selection area box.
Repeat for each image.
Interpreting the Diagnostic Information
A few things to note here when looking at the diagnostics information:
Calibration that reports a high RMS error (above 1), with many red blocks, and/or a small number of points is not recommended for use as it could result in poor quality depth or animation data.
If there are detected points outside of the face area (area of interest), they can be considered as outliers. These points, while valid, have no impact on the depth generation process and animation. Note, selecting an area of interest helps by showing only the points in that area.
No points detected, or a really low number of points detected means that all points that could be detected are filtered out by Feature Match Error threshold. If they are filtered out, it means that their error is large and we can consider this calibration as bad for the current footage.
An example of good calibration if the RMS is low (close to 1) and there are no red areas in the area of interest.
The RMS reprojection error in the Calibration Diagnostic tool is different from the one in the Calibration Generator. The one in the diagnostic tool will most likely have a higher bound than the one in the generator tool. This is due to the generator tool being more precise with detecting the corners of the checkerboard.
Reprojection Error represents the distance between a projected point (point projected using the generated calibration) and a measured point (the corner of the checkerboard, or features of the face).
Once this distance is determined, the RMS, Mean, and Median are measured between all points. Each of these is its own mathematical information that helps with the diagnostics.
RMS (root mean squared) error is calculated by summing the squares of all the Reprojection Errors, divided by the number of errors and then calculating the square root of it.
Mean is calculated by summing all the Reprojection Errors and dividing by the number of errors.
Median is calculated by sorting all the Reprojection Errors, and then picking the element in that sorted array which is at the position (n+1)/2 or (n/2 + (n/2 + 1))/2, depending if the number of errors is odd or even.
| Property | Description |
|---|---|
Camera Calibration | Select a camera calibration asset (created by the Calibration Generator) to use with this performance take. |
RMS Error Threshold | An upper bound value under which the RMS is considered acceptable. 0 < RMS Error < RMS Error Threshold. |
Feature Match Error Threshold | An upper bound value used to discard outliers during the feature detection process. In cases where the calibration is poor, this threshold can hide some valid points (for which the error is too high) - so in some cases where only a few points are detected it might be necessary to increase this number. |
Reprojection Error Threshold | An upper-bound value used for which the Reprojection error is considered to be good. As you can see, this value is read-only and therefore may differ from RMS. In the tool, this value is used to draw the detected points and projected points and to color-code the reprojection error (distance) between them. |