Timed Data Monitor

Unreal Engine can ingest a variety of data types from multiple sources at the same time. For example, in Virtual Production, the engine can receive captured frames from the camera on SDI as well as the position and orientation of the camera from the tracking system through Live Link. The Timed Data Monitor is a solution to configure and visualize how all this incoming timed data relates to each other and the engine's time.

The Timed Data Monitor currently supports sources from:

• Live Link

• SDI

You can extend the functionality of Timed Data Monitor to include other timed data sources through code. Use the TimedDataSource framework to register your custom data source with Timed Data Monitor.

The sections below cover how to use the Timed Data Monitor and the settings you can modify for your project.

Getting Started with Timed Data Monitor

Follow these steps to use the Timed Data Monitor with your project.

1. In the Editor's main menu, choose Edit > Plugins to open the Plugins window.

2. In the Plugins window, find the Timed Data Monitor plugin in the Virtual Production section.

   

3. Check the Enabled box.

4. In the popup, choose Yes.

   

5. Restart the Editor when prompted.

   

6. In the Editor's main menu, choose Window > Developer Tools > Timed Data Monitor to open the Timed Data Monitor window.

   

Timecode and Genlock

At the top of the Timed Data Monitor, you can view the CustomTimestep and TimecodeProvider used in the project. Without needing to switch to another window, you can also configure an offset for the TimecodeProvider.

For SDI inputs, these settings are changing the settings of the opened stream. It's not changing the actual MediaSource asset that was used when opening the input. If you find a good set of options, consider changing the associated asset.

Sources and Channels

Once the Timed Data Monitor plugin is enabled in Unreal Engine, any Live Link, SDI, or custom sources using the TimedDataSource framework in the project will automatically appear in the tool. Each source can contain multiple channels that are viewable by clicking the arrow in front of the source name. The table below describes the information represented in the UI for each source.

Buffer Visualizer

The Buffer Visualizer is a separate window to show a visualization of all the channels together. This is a larger, more comprehensive view of what's shown for each channel in the Timing Diagram column.

The buffers are represented with the following UI elements:

• Vertical Line: Represents the evaluation time. The line is green when inside the buffer, and red when it's outside the available samples.

• Light gray rectangle: Represents the current timed samples available in the channel.

• Dark gray rectangle: Represents the average of the timed samples available in the channel.

• White rectangle: Represents the standard deviation of the oldest timed sample and newest timed sample available.

The bigger the white rectangle is, the more variation or jitter in the sample's buffer. When you see this larger variation, you will probably need to allow more buffer between the evaluation time and the newest sample available to be sure you are always able to evaluate a sample at the desired time.

To open the Buffer Visualizer in the Timed Data Monitor:

• Select the Open a buffer visualizer button at the top of the Timed Data Monitor window.

  

Calibration

The Timed Data Monitor includes functionality to calibrate all the enabled sources to find an alignment point. Calibration can be initiated from the Timed Data Monitor UI or through Blueprints.

During calibration, the Timed Data Monitor will modify the global offset of the TimecodeProvider and buffer sizes of the sources so that all channels have samples in the range of the evaluation time. By default, the standard deviation of time differences will be used to find a spot with more statistical assurance that it will always be aligned. The algorithm can be modified from the project settings.

Calibration Example

In the video, there are multiple LiveLink sources connected but not aligned. In the viewport, the actors are out of sync. In the Buffer Visualization window, you can see the misalignment: some sources have their samples too early; some contain the evaluation time; and some that are too late. One source LLTS.5 has a smaller buffer amount than the others.

When the calibration process starts, the algorithm:

• Grows the LLTS.5 buffer size to be able to align it with the other samples.

• Modifies the global offset on the TimecodeProvider to get all sources aligned. The offset selected is the smallest required to align LLTS.0 with the other samples since it was the furthest from the evaluation point.

Once calibration finishes, the buffers are all in the range of the evaluation time and all the actors are now in sync.

Timed Data Monitor State

In the top right corner of the Timed Data Monitor is a visualization of the global state of the incoming data and how it relates to the engine's time. The possible states include:

The example below shows how the sources can affect the global state of the monitor. In the example, there are two Live Link sources:

• The Time Correction for the first source is set to -0.1. This puts the source out of sync with the engine's time, as represented by the red vertical line in the Timing Diagram.

• The Time Correction for the second source is set to 0.1. This puts the source in sync with the engine's time, as represented by the green vertical line in the Timing Diagram.

When both sources are enabled, the global state is set to Outside Range, because the first source is out of sync with the engine's time. Disabling the first source switches the state to Synchronized, since only the second source was in sync with the engine's time.

Plugin Settings Reference

In the Plugin settings, there are many ways to customize the calibration process and the functionality of Timed Data Monitor in general. The table below describes all the settings and how they impact functionality in the tool.

Calibration

UI