Navigation
API > API/Plugins > API/Plugins/TmvMedia
This struct describes information (memory layout, color model, component type) about a mipmap buffer for a video frame. A complete TMV frame for a given time index will be composed of multiple mips each of which is described by this struct. The goal is to allow decoders to provide the sample conversion implementation enough information to prepare mip frame buffers with the needed layout for the decoder, and then be able to handle the conversion on the GPU through PS or CS shaders.
Memory Surface Layout: The pixel components may be packed in a single memory plane, separated in different planes or mixed (semi-planar ex: NV12).
Chroma subsampling for YUV color model is supported by having planes with different dimensions.
The memory data layout can be either tiled or scanline. If the layout is tiled, the plane definitions (dimensions, stride, padding, etc) are per tile rather than for the whole frame.
FOURCC is not enough: There are FOURCC types for fully planar YUV types such as P010 (420 10 bits) or P210 (422, 10 bits). However, those formats don't describe custom padding for decoder tile or macro block memory alignment for best performance. This is why at this level of the api, it is preferable to not try to align with existing FOURCC. The current descriptor is still probably incomplete as it only targets a subset of implementations, but it is expected to evolve over time.
Some reference on the subject:
- https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h -> FOURCC + Modifier to account for memory layout.
Component Type: All components have the same type, either float or int.
Color Model: Defines the meaning of the first 3 components: RGB, YUV Alpha is supported as the 4th component in the sampler converter, regardless of the color model.
| Name | FTmvMediaFrameMipInfo |
| Type | struct |
| Header File | /Engine/Plugins/Media/TmvMedia/Source/TmvMedia/Public/TmvMediaFrameInfo.h |
| Include Path | #include "TmvMediaFrameInfo.h" |
Syntax
struct FTmvMediaFrameMipInfo
Variables
Public
| Name | Type | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|---|
| ColorInfo | FTmvMediaFrameColorInfo | Specifies the color information (color space, encoding, yuv matrix and range) for the given frame. | TmvMediaFrameInfo.h | |
| ColorModel | ETmvMediaFrameColorModel | Indicate the color model that allow to determine how the component should be interpreted. | TmvMediaFrameInfo.h | |
| Height | int32 | Number of vertical samples in this mip. | TmvMediaFrameInfo.h | |
| Layout | ETmvMediaFrameBufferLayout | Defines the memory layout of all mip planes. | TmvMediaFrameInfo.h | |
| MipLevel | int32 | Mip level | TmvMediaFrameInfo.h | |
| NumComponents | int32 | Total number of components per sample. May be packed in planes. | TmvMediaFrameInfo.h | |
| NumTiles | FIntPoint | Number of tiles on each dimensions in this mip. | TmvMediaFrameInfo.h | |
| Planes | TArray< FTmvMediaFramePlaneInfo, TInlineAllocator< 4 > > | Description of memory planes. | TmvMediaFrameInfo.h | |
| TileHeight | int32 | Number of vertical samples in a tile. | TmvMediaFrameInfo.h | |
| TileWidth | int32 | Number of horizontal samples in a tile. | TmvMediaFrameInfo.h | |
| Width | int32 | Number of horizontal samples in this mip. | TmvMediaFrameInfo.h |
Functions
Public
| Name | Remarks | Include Path | Unreal Specifiers |
|---|---|---|---|
SIZE_T GetAllPlaneMemorySizeInBytes () |
Returns the memory size in bytes for all planes. | TmvMediaFrameInfo.h | |
SIZE_T GetMemorySizeInBytes() |
Returns the total memory size in bytes. | TmvMediaFrameInfo.h | |
bool GetPlaneBufferOffset
(
int32 InPlaneIndex, |
Gets the plane buffer offset accounting for the size of all previous planes. | TmvMediaFrameInfo.h | |
int32 GetPlaneIndexForComponent
(
int32 InComponentIndex, |
Finds the plane index for the given component. | TmvMediaFrameInfo.h |