Temporal Super Resolution

Temporal Super Resolution (TSR) is a platform-agnostic Temporal Upscaler that enables Unreal Engine to render beautiful 4K images. Images come at a fraction of the cost by amortizing some of the costly rendering calculations across many frames. It does this by rendering lower internal resolution than what Temporal Anti-Aliasing Upsampling (TAAU) in Unreal Engine 4 could do.

TSR provides a native high-quality upsampling technique to meet the demand of next-generation games. It realizes the possibilities needed with the fidelity and details required by Nanite geometry while allowing rendering frames at much lower resolution to accommodate enough performance for Lumen.

The comparison below demonstrates the quality and performance difference between frames rendered at native 4K and those rendered at 1080p upscaled to 4K. With TSR, it's possible to achieve image quality near 4K resolution while also reducing GPU frame time by half.

Each of the images in the comparison above are 4K images that are limited to the width of this page. To see their fully uncompressed resolution, right-click on either and save them to your computer, or open them in a new browser window.

Temporal Super Resolution has the following properties:

• Rendered frames approach the quality of native 4K with input resolutions as low as 1080p.
• Less "ghosting" artifacts against high-frequency backgrounds than was visible with Unreal Engine 4's default Temporal Anti-Aliasing method.
• Reduced flickering on geometry with high complexity.
• Supports Dynamic Resolution scaling on console platforms.
• Runs on any hardware that supports D3D11, D3D12, Vulkan, Metal, PlayStation 5, and Xbox Series S | X.
• Shaders are optimized specifically for PlayStation 5 and Xbox Series S | X GPU architectures.

In the rendering chain, TSR happens after the depth of field and everything that follows is upscaled.

TSR's Scalability

You can use Temporal Super Resolution across supported platforms, but you will want to customize the upscaling settings of each platform based on the needs of your project. Explore the sections below to learn some of the ways you can go about checking TSR in your project and then scaling it appropriately.

Understanding the Caveats of Temporal Accumulation of Details

Advertising only the increases to frame rate that TSR allows by rendering at lower resolutions wouldn't entirely be fair since temporal upscalers in general have their own caveats. Given enough frame time with a static image, any temporal upscaler could render identical results. Also like any temporal upscaler, TSR accumulates lower resolution frames over time to converge an image, and the detail of an image can only be known after enough details are accumulated. For instance, how thick any geometry is cannot be known in the first frame.

The rendering resolution is controlled by the Screen Percentage. This controls how much information is available for one frame. The rest of the information needed to converge depends on the remaining parts of the frame to be rendered. This means that information displayed by TSR is dependent on both the resolution of the frame being rendered and the frame rate because it influences how fast details can accumulate.

More than just the GPU can limit frame rate that affects TSR, like when CPU bound or VSync on non-variable refresh rate monitors.

This is where the console command stat tsr can help identify what may affect TSR's accumulation to the overall image. When called, the command adds two stats to the ones normally displayed when calling stat unit. These are TSR feed and TSR1spp.

TSR feed shows the number of pixels per second being fed into TSR, which is an important macroscopic metric to understand how much data TSR has to converge to the overall image. Rendering Resolution Width * Rendering Resolution Height * FrameRate = Display Resolution Width * Display Resolution Height * ScreenPercentage^2 * FrameRate. The benefit of this metric is it indicates the macroscopic image quality in motion regardless of the display resolution. To illustrate how this scales:

TSR 1spp is TSR's convergence rate — the time needed for TSR to have enough data to reach one sample per pixel. It is especially important in motion where there can be disocclusion that needs to accumulate details quickly. TSR 1spp = 1000 / (ScreenPercentage^2 * FrameRate).

An example situation of how to use this data would be if your Screen Percentage were set to 50, this means you need (50/100)^-2 = 4 frames to have at least one sample per pixel. If each frame takes 16.6 milliseconds to render, it means a disocclusion area on screen will take four times that to have enough data or 4*16.6 = 66.4ms.

TSR's Upscaling GPU Cost

The primary goal of TSR is upscaling. The majority of its GPU work scales based on the resolution it is given. That is due to some of the GPU cost of TSR needing to be done at a higher display resolution than the rendering resolution.

Open the GPU statistical display from the console using stat gpu. With this open, you can adjust the primary Screen Percentage to see the difference in performance when rendering at 100 percent versus 50% screen percentage, like the examples taken below from the Valley of the Ancients sample project.

The parallax heuristic of TSR depends on the depth and velocity buffer rather than the scene color and translucencies of the frame because the depth and velocity buffers often finish on the GPU much earlier than the scene color and translucency buffers do. This allows the entire TSR parallax heuristic to compute asynchronously on the GPU, filling in the gaps where the GPU is underused by other rendering algorithms with r.TSR.AsyncCompute=2.

In Fortnite Chapter 4 on PlayStation 5 and Xbox Series X, this is offsetting approximately 0.5ms of TSR's total GPU cost when testing performance over an entire Battle Royal performance replay. In the replay, about 0.1ms is saved, bringing down the effective TSR cost to 1.5ms and critical path GPU cost to finish rendering the frame to 1.1ms.

Console Variables

The Engine Scalability Settings scale the quality of TSR across all platforms. They use predefined scalability groups from Low to Cinematic. Each group is defined by a set of console variables that affect different rendering properties, including anti-aliasing.

You can inspect how the scalability settings are used by opening the BaseScalability.ini file in the [Unreal Engine Root]/Engine/Config folder. Look at the AntiAliasingQuality sections to see how TSR is scaled depending on the anti-aliasing quality group being used. You can modify the anti-aliasing quality groups for your own project by changing the settings in [Your Project Root]\Config\DefaultScalability.ini.

For example, an adjustment you may want to make for your project is that TSR includes a spatial anti-aliasing algorithm that is used whenever the history needs to be rejected to avoid stale data that could introduce ghosting errors to the current frame. Game projects that still want to render at a higher resolution, or higher frame rate, may not need this option since the visible aliasing may not be subject to rejection. History rejection for TSR is controlled with r.TSR.RejectionAntiAliasingQuality.

Hidden Additional GPU Costs of Temporal Upscaling

Temporal Super Resolution and any other temporal upscaler happens in the middle of the post-processing chain of the renderer. This means that passes, such as Motion Blur or Tonemapper, running after TSR are no longer scaling with the primary Screen Percentage. In the absence of a secondary screen percentage with a spatial upscaler (r.SecondaryScreenPercentage.GameViewport), the output resolution of TSR runs exactly at the display resolution rather than the rendering resolution. With TSR enabled by default, a lot of effort is focused on reducing the hidden temporal upscaling costs of passes that happen after TSR.

Motion Blur

Motion Blur optimizations combine for 3x motion blur GPU performance cost improvements on PlayStation 5 and Xbox Series X, platforms that always display 2160p backbuffer, even on lower resolution televisions.

• Increased TSR costs when using stat gpu:
  • TSR cost may increase slightly when using stat gpu when comparing Motion Blur enabled and disabled.
  • When Motion Blur is enabled, TSR takes over its Velocity Flatten pass that normally runs at input resolution. This is controlled using the console command r.MotionBlur.AllowExternalVelocityFlatten, and it is enabled by default.
  • TSR outputs half-resolution Scene Color to reduce bottlenecks in memory bandwidth that can occur in large directional blur kernels. This is controlled using the console command r.MotionBlur.HalfResInput, and it is enabled by default.
• Half-resolution on directional blur:
  • Directional blur happening at display resolution is optimized to automatically run at half-resolution on very large movements. Enabling this feature will reduce VALU cost of motion blur.
  • Enable this optimization using the console command r.MotionBlur.HalfResGather 1.
• Motion Blur Half and Quarter Resolution:
  • TSR and Motion Blur are able to output half or quarter resolution, This is of importance for Gaussian and Convolution Bloom, Lens Flare, Eye Adaptation (Auto Exposure), and Local Exposure that all run after Motion Blur.

The PostProcessQuality scalability group scales some of these r.MotionBlur.* in Engine/Config/BaseScalability.ini.

Additional Notes

Troubleshooting TSR

With TSR being a temporal upscaler, there are times you'll need to diagnose artifacts that affect temporal upscalers. You can use the VisualizeTemporalUpscaler showflag to look at the raw input and output buffers as a starting point.

You can access this show flag from the Level Editor's Show > Visualize menu by selecting Temporal Upscaler (TSR, TAAU, or third party plugins).

For more information, see Temporal Upscalers.

Adjusting for Pixel Flickering

The flickering of pixels in the frame can happen for various reasons and sometimes can be mitigated. There are console variables to define the difference between pixel flickering, like when looking at a building at a grazing angle, or in a pixel shader animation.

You can use r.TSR.ShadingRejection.Flickering.Period to set the period in 60hz frames in which luma oscillations at equal or greater frequency are considered flickering and should be stabilized. But this stabilization can introduce ghosting when it happens on undesired areas of the image

This setting of TSR is important because it is what differentiates a pixel flickering from other desired visual effects, like pixel shader animation. However, an important difference is that flickering happens regardless of frame rate. Visual effects that are based on time and are therefore independent of the frame rate. This can mean that a visual effect that looks smooth at 60hz might appear to "flicker" at lower frame rates, like 24hz.

It's important that visuals authored by an artist remain unaffected by frame rate since TSR scales to meet GPU performance. This can have the unintended effect of altering visuals when a player's machine is running at lower than expected frame rates. For this purpose, the console command r.TSR.ShadingRejection.Flickering.Period automatically reduces anti-flickering when the frame rate drops lower than the frame rate defined with r.TSR.ShadingRejection.Flickering.FrameRateCap (default is 60hz). This means that geometric details that might be stable at 60hz might become less stable at a lower frame rate. This behavior is enabled by default but can be opted out of with r.TSR.ShadingRejection.Flickering.AdjustToFrameRate.

Supported Platforms

Temporal Super Resolution is available on desktop renderers across all desktop hardware that supports Shader Model 5. Temporal Super Resolution is supported on the following platforms:

• Windows D3D11 SM5, D3D12 SM5, D3D12 SM6, and Vulkan SM5
• Linux Vulkan SM5
• Mac Metal SM5
• PlayStation 5 and Xbox Series S | X

TSR's upscaling quality and behavior is strictly identical across all supported platforms. However, TSR has been specifically optimized for AMD RDNA GPUs used in PlayStation 5 and Xbox Series S | X consoles, taking advantage of 16-bit types and packed instructions.

Console Variables