1. Epic GamesDeveloper
  3. UMediaCapture

UMediaCapture

Abstract base class for media capture.

API > API/Plugins > API/Plugins/MediaIOCore

Abstract base class for media capture.

MediaCapture capture the texture of the Render target or the SceneViewport and sends it to an external media device. MediaCapture should be created by a MediaOutput.

   
Name UMediaCapture
Type class
Header File /Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Public/MediaCapture.h
Include Path #include "MediaCapture.h"

Syntax

UCLASS (MinimalAPI, Abstract, EditInlineNew, BlueprintType, HideCategories=(Object))  
class UMediaCapture : public UObject

Inheritance Hierarchy

Derived Classes

UMediaCapture derived class hierarchy

Constructors

Name Remarks Include Path Unreal Specifiers

UMediaCapture()

   MediaCapture.h  

Destructors

Name Remarks Include Path Unreal Specifiers

virtual ~UMediaCapture()

 Default constructor / destructor to use forward declared uniqueptr MediaCapture.h  

Structs

Name Remarks
FCaptureBaseData  
FMediaCaptureResourceData Args passed to OnFrameCaptured
FMediaCaptureSyncData  
FQueuedCaptureData  

Constants

Name Type Remarks Include Path
MaxCaptureDataAgeInFrames int32   MediaCapture.h

Variables

Public

Name Type Remarks Include Path Unreal Specifiers
OnOutputSynchronization FMediaCaptureOutputSynchronization Called when output data is ready to be sent out. MediaCapture.h  
OnStateChanged FMediaCaptureStateChangedSignature Called when the state of the capture changed. MediaCapture.h
  • BlueprintAssignable
  • Category="Media|Output"
OnStateChangedNative FMediaCaptureStateChangedSignatureNative Called when the state of the capture changed. MediaCapture.h  

Protected

Name Type Remarks Include Path Unreal Specifiers
AccessingCapturingSource FCriticalSection   MediaCapture.h  
bIsAutoRestartRequired std::atomic< bool >   MediaCapture.h  
bLoggedOutputNotReady bool Used to avoid repeatedly logging while the underlying sdk is not ready to output. MediaCapture.h  
bPendingOutputResourcesInitialized std::atomic< bool >   MediaCapture.h  
bShouldCaptureRHIResource std::atomic< bool >   MediaCapture.h  
bSyncHandlersInitialized std::atomic< bool >   MediaCapture.h  
bUseRequestedTargetSize bool   MediaCapture.h  
bViewportHasFixedViewportSize bool   MediaCapture.h  
bWaitingForStopCapture bool This indicates the capture is in the process of stopping and is waiting for StopCapture_Future's result to be ready. MediaCapture.h  
CaptureDataQueue TArray< FQueuedCaptureData >   MediaCapture.h  
CaptureDataQueueCriticalSection FCriticalSection Used to synchronize access to the capture data queue. MediaCapture.h  
CaptureRenderPipeline TPimplPtr< UE::MediaCapture::FRenderPipeline > Holds the render passes that a texture will go through during media capture. MediaCapture.h  
CaptureRequestCount int32   MediaCapture.h  
CaptureSource TSharedPtr< UE::MediaCapture::Private::FCaptureSource, ESPMode::ThreadSafe > Capture source MediaCapture.h  
ConversionOperation EMediaCaptureConversionOperation   MediaCapture.h  
DesiredCaptureOptions FMediaCaptureOptions   MediaCapture.h  
DesiredOutputBufferDescription FRDGBufferDesc The BufferDesc used to create the final output buffer that will be readback to the output. MediaCapture.h  
DesiredOutputPixelFormat EPixelFormat   MediaCapture.h  
DesiredOutputResourceType EMediaCaptureResourceType   MediaCapture.h  
DesiredOutputTextureDescription FRDGTextureDesc The TextureDesc used to create the final output texture that will be readback to the output. MediaCapture.h  
DesiredPixelFormat EPixelFormat   MediaCapture.h  
DesiredSize FIntPoint   MediaCapture.h  
FrameManager TPimplPtr< UE::MediaCaptureData::FFrameManager >   MediaCapture.h  
MediaOutputName FString   MediaCapture.h  
MediaState std::atomic< EMediaCaptureState >   MediaCapture.h  
PendingCommands TArray< TFuture< void > >   MediaCapture.h  
PendingFrameCount std::atomic< int32 >   MediaCapture.h  
SyncHandlers TArray< TSharedPtr< FMediaCaptureSyncData > > Array of sync handlers (fence) to sync when captured buffer is completed MediaCapture.h  
SyncPointWatcher TPimplPtr< UE::MediaCaptureData::FSyncPointWatcher > Watcher thread looking after end of capturing frames MediaCapture.h  
ViewExtension TSharedPtr< class FMediaCaptureSceneViewExtension > Holds a view extension that callbacks MediaCapture.h  
WaitingForRenderCommandExecutionCounter std::atomic< int32 >   MediaCapture.h  

Functions

Public

Name Remarks Include Path Unreal Specifiers
Stop the current capture if there is one. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

void CaptureImmediate_RenderThread ( FRDGBuilder& GraphBuilder,
FRHITexture* InSourceTexture,
FIntRect SourceViewRect
)

   MediaCapture.h  

void CaptureImmediate_RenderThread ( FRDGBuilder& GraphBuilder,
FRDGTextureRef InSourceTextureRef,
FIntRect SourceViewRect
)

   MediaCapture.h  

bool CaptureRHITexture ( const FRHICaptureResourceDescription& ResourceDescription,
FMediaCaptureOptions CaptureOptions
)

 Stop the actual capture if there is one. MediaCapture.h  

bool CaptureSceneViewport ( TSharedPtr< FSceneViewport >& SceneViewport,
FMediaCaptureOptions CaptureOptions
)

 Stop the actual capture if there is one. MediaCapture.h  
Stop the actual capture if there is one. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

const FAudioDeviceHandle & GetCaptureAudioDevice()

 Get the audio device to capture. MediaCapture.h  

const FMediaCaptureOptions & GetDesiredCaptureOptions()

 Get the capture options specified by the user. MediaCapture.h  

EPixelFormat GetDesiredPixelFormat()

 Get the desired pixel format of the current capture. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

FIntPoint GetDesiredSize()

 Get the desired size of the current capture. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

FString GetMediaOutputName()

 Get the name of the media output that created this capture. MediaCapture.h  

virtual const FMatrix & GetRGBToYUVConversionMatrix ()

 Get the RGB to YUV Conversion Matrix, should be overriden for media outputs that support different color gamuts (ie. BT 2020). MediaCapture.h  

virtual EMediaCaptureState GetState()

 Get the current state of the capture. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

virtual bool HasFinishedProcessing()

 Check whether this capture has any processing left to do. MediaCapture.h  
Returns true if media capture implementation supports synchronization logic. MediaCapture.h  

virtual bool PostInitializeCaptureRenderTarget ( UTextureRenderTarget2D* InRenderTarget
)

 Called after initialize for render target capture type MediaCapture.h  

virtual bool PostInitializeCaptureRHIResource ( const FRHICaptureResourceDescription& InResourceDescription
)

 Called after initialize for rhi resource capture type MediaCapture.h  

virtual bool PostInitializeCaptureViewport ( TSharedPtr< FSceneViewport >& InSceneViewport
)

 Called after initialize for viewport capture type MediaCapture.h  

void ResetFixedViewportSize ( TSharedPtr< FSceneViewport > InViewport,
bool bInFlushRenderingCommands
)

   MediaCapture.h  

bool SetCaptureAudioDevice ( const FAudioDeviceHandle& InAudioDeviceHandle
)

 Set the audio device to capture. MediaCapture.h  

void SetFixedViewportSize ( TSharedPtr< FSceneViewport > InSceneViewport,
FIntPoint InSize
)

   MediaCapture.h  

void SetMediaOutput ( UMediaOutput* InMediaOutput
)

 Set the media output. Can only be set when the capture is stopped. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

virtual void SetValidSourceGPUMask ( FRHIGPUMask GPUMask
)

 Set the valid GPU mask of the source buffer/texture being captured. Can be called from any thread. MediaCapture.h  

void StopCapture ( bool bAllowPendingFrameToBeProcess
)

 Stop the previous requested capture. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

bool TryCaptureImmediate_RenderThread ( FRDGBuilder& GraphBuilder,
FRHITexture* InSourceTexture,
FIntRect SourceViewRect
)

 Captures a resource immediately from the render thread. Used in RHI_RESOURCE capture mode MediaCapture.h  

bool TryCaptureImmediate_RenderThread ( FRDGBuilder& GraphBuilder,
FRDGTextureRef InSourceTextureRef,
FIntRect SourceViewRect
)

   MediaCapture.h  

virtual bool UpdateAudioDeviceImpl ( const FAudioDeviceHandle& InAudioDeviceHandle
)

   MediaCapture.h  

virtual bool UpdateRenderTargetImpl ( UTextureRenderTarget2D* InRenderTarget
)

   MediaCapture.h  

bool UpdateSceneViewport ( TSharedPtr< FSceneViewport >& SceneViewport
)

 Update the current capture with a SceneViewport. MediaCapture.h  

virtual bool UpdateSceneViewportImpl ( TSharedPtr< FSceneViewport >& InSceneViewport
)

   MediaCapture.h  
Update the current capture with every frame for a TextureRenderTarget2D. MediaCapture.h
  • BlueprintCallable
  • Category="Media|Output"

Overridden from UObject

Name Remarks Include Path Unreal Specifiers

virtual void BeginDestroy()

   MediaCapture.h  

virtual FString GetDesc()

   MediaCapture.h  

Protected

Name Remarks Include Path Unreal Specifiers

virtual void BeforeFrameCaptured_RenderingThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FBufferRHIRef InBuffer
)

 Called at the beginning of the Capture_RenderThread call if output resource type is Buffer MediaCapture.h  

virtual void BeforeFrameCaptured_RenderingThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FTextureRHIRef InTexture
)

 Called at the beginning of the Capture_RenderThread call if output resource type is Texture MediaCapture.h  

virtual bool CaptureRenderTargetImpl ( UTextureRenderTarget2D* InRenderTarget
)

   MediaCapture.h  

virtual bool CaptureSceneViewportImpl ( TSharedPtr< FSceneViewport >& InSceneViewport
)

   MediaCapture.h  
Capture the data that will pass along to the callback. MediaCapture.h  

FString GetCaptureSourceType()

   MediaCapture.h  

TSharedPtr< FSceneViewport > GetCapturingSceneViewport()

   MediaCapture.h  

EMediaCaptureConversionOperation GetConversionOperation()

   MediaCapture.h  

virtual FRDGBufferDesc GetCustomBufferDescription ( const FIntPoint& InDesiredSize
) const

 Get the desired output buffer description when the conversion is custom and operating in buffer resource type MediaCapture.h  

virtual EPixelFormat GetCustomOutputPixelFormat ( const EPixelFormat& InPixelFormat
) const

 Get the pixel format of the buffer when the conversion operation is custom. MediaCapture.h  
Get the desired output resource type when the conversion is custom. MediaCapture.h  

virtual FIntPoint GetCustomOutputSize ( const FIntPoint& InSize
) const

 Get the size of the buffer when the conversion operation is custom. MediaCapture.h  
Get the output texture's flags. MediaCapture.h  

UTextureRenderTarget2D * GetTextureRenderTarget()

   MediaCapture.h  

virtual bool InitializeCapture()

 Initialization method to prepare implementation for capture MediaCapture.h  

virtual bool IsReadyToOutput ()

 Implementations can return false when the output sdk is not ready. MediaCapture.h  

virtual void LockDMATexture_RenderThread ( FTextureRHIRef InTexture
)

   MediaCapture.h  

virtual void OnCustomCapture_RenderingThread ( FRDGBuilder& GraphBuilder,
const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FRDGTextureRef InSourceTexture,
FRDGTextureRef OutputTexture,
const FRHICopyTextureInfo& CopyInfo,
FVector2D CropU,
FVector2D CropV
)

 Callback when the source texture is ready to be copied on the GPU. MediaCapture.h  

virtual void OnCustomCapture_RenderingThread ( FRDGBuilder& GraphBuilder,
const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FRDGTextureRef InSourceTexture,
FRDGBufferRef OutputBuffer,
const FRHICopyTextureInfo& CopyInfo,
FVector2D CropU,
FVector2D CropV
)

 Callback when the source texture is ready to be copied on the GPU. MediaCapture.h  

virtual void OnFrameCaptured_AnyThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
const FMediaCaptureResourceData& InResourceData
)

 Callback when the buffer was successfully copied to CPU ram. MediaCapture.h  

virtual void OnFrameCaptured_RenderingThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
void* InBuffer,
int32 Width,
int32 Height,
int32 BytesPerRow
)

 Callback when the buffer was successfully copied to CPU ram. MediaCapture.h  

virtual void OnRHIResourceCaptured_AnyThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FBufferRHIRef InBuffer
)

   MediaCapture.h  

virtual void OnRHIResourceCaptured_AnyThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FTextureRHIRef InTexture
)

 AnyThread version of the above callbacks. MediaCapture.h  

virtual void OnRHIResourceCaptured_RenderingThread ( FRHICommandListImmediate& RHICmdList,
const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FTextureRHIRef InTexture
)

 Callbacks when the buffer was successfully copied on the GPU ram. MediaCapture.h  

virtual void OnRHIResourceCaptured_RenderingThread ( FRHICommandListImmediate& RHICmdList,
const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FBufferRHIRef InBuffer
)

   MediaCapture.h  

virtual void OnRHITextureCaptured_RenderingThread ( const FCaptureBaseData& InBaseData,
TSharedPtr< FMediaCaptureUserData, ESPMode::ThreadSafe > InUserData,
FTextureRHIRef InTexture
)

   MediaCapture.h  

virtual void PostCaptureCleanupImpl ()

 Override this to cleanup resources in the capture implementation after StopCapture is done. MediaCapture.h  

void RestartCapture()

   MediaCapture.h  

void SetState ( EMediaCaptureState InNewState
)

   MediaCapture.h  

virtual bool ShouldCaptureRHIResource()

 Should we call OnFrameCaptured_RenderingThread() with a RHI resource -or- readback the memory to CPU ram and call OnFrameCaptured_RenderingThread(). MediaCapture.h  

virtual bool ShouldCaptureRHITexture()

   MediaCapture.h  

virtual bool ShouldCaptureThisFrame ( const FCaptureBaseData& InBaseData
) const

 Allows capture implementaion to decide whether to discard a frame based on its base data. MediaCapture.h  

virtual TFuture< void > StopCapture_Future ()

 Version of StopCaptureImpl that returns a future. MediaCapture.h  

virtual void StopCaptureImpl ( bool bAllowPendingFrameToBeProcess
)

   MediaCapture.h  

virtual bool SupportsAnyThreadCapture ()

 Whether the capture callbacks can be called on any thread. MediaCapture.h  

virtual bool SupportsAutoRestart()

 Whether the capture implementation supports restarting on input format change. MediaCapture.h  

virtual bool SupportsStopCapture_Future()

 Returns whether the capture implementation supports the implementation of StopCapture that returns a future. MediaCapture.h  

virtual void UnlockDMATexture_RenderThread ( FTextureRHIRef InTexture
)

   MediaCapture.h  
Whether capture should be done using experimental scheduling. MediaCapture.h  

virtual bool ValidateMediaOutput()

   MediaCapture.h  

virtual void WaitForGPU ( FRHITexture* InRHITexture
)

 When using GPUTextureTransfer, calling this will wait for a signal set by the underlying library. MediaCapture.h  

Static

Name Remarks Include Path Unreal Specifiers

static int32 GetBytesPerPixel ( EPixelFormat InPixelFormat
)

 Returns bytes per pixel based on pixel format MediaCapture.h  

static const TSet< EPixelFormat > & GetSupportedRgbaSwizzleFormats()

   MediaCapture.h  

Ask questions and help your peers Developer Forums

Write your own tutorials or read those from others Learning Library

On this page