Subtitles and Closed Captions Plugin

What is it?

An Unreal Engine plugin that is designed to support subtitles, closed captions, and scene descriptions decoupled from audio. For example, these new subtitles can be cued from any other system, independent of any related audio asset duration or playback state. This new subtitle system will ultimately supersede and deprecate the existing system which does not support independent use of subtitles from sounds. One may customize subtitles' display timing, fonts, colors, etc.

Where is it?

\Engine\Plugins\Experimental\SubtitlesAndClosedCaptions

Include: #include SubtitlesSubsystem.h

To load the plugin in the Editor, go to Edit > Plugins, in the tab that opens type the name or scroll to Subtitles and Closed Captions, select the checkbox, and click Restart the Editor.

How Do I Use It?

Each subtitle is stored as a USubtitleAssetUserData UCLASS derived from UAssetUserData and can be subclassed to store additional custom data on any Unreal asset object.

USubtitleAssetUserData Members

Text: The localizable FText to appear in the subtitle.

• Multiple lines of text are supported using Shift + Enter or by including \n in the text to specify new lines.

• The FTexts are localized to the current language on the client using the localization system's hashkey implementation – localization occurs before and independent of the subtitle system.

Duration: Time to display subtitle in seconds.     

• Default: 3 seconds.

• Must be >= 0.05 seconds.

• Only enabled when Duration Type is set to UseDurationProperty.

Duration Type: Can be set to UseDurationProperty, which enables the Duration property, or to UseSoundDuration to automatically stop when an attached sound finishes.  

• UseSoundDuration only works when the subtitle is added as an AssetUserData to a SoundBase asset.

StartOffset: Time offset in seconds to start displaying the subtitle line.  

• This may be used to simplify displaying multiple subtitles per audio asset, e.g. a long SoundWave with multiple sentences could use multiple subtitles, each with StartOffset set to the time offset of the start of each sentence.

• Default: 0 seconds.

Priority: If multiple subtitles are played concurrently, only the ones with highest (largest value) priority will play instead of those with lower values.

• This applies even if the subtitles only partially overlap. Overlap is determined from StartOffset and duration.

• If a new subtitle is set to display immediately or via StartOffset while a higher priority subtitle is being displayed, the lower priority subtitle will not be displayed.

• Only one of each category (subtitle, audio description, etc.) will display at a time.

• If two competing subtitles have the same priority, the oldest one queued will continue to display. Whenever a queued subtitle isn’t displayed due to a priority conflict, a warning is sent to the debug log.

• Default: 1.

ESRB: ESRB rating of category of subtitle. This is a descriptive field set and used by game developers.  

• This enables the caller to determine if a subtitle line should be displayed by checking this value. 

• Default: ESRB::Everyone.

SubtitleType: Specifies if the asset is a Subtitle, ClosedCaption, or AudioDescription.  This is a descriptive field set by game developers.

• This enables the caller to determine if a subtitle line should be displayed by checking this value.

• Each SubtitleType has independent display parameters set via a TextBlock widget documented below.

• When displayed, different subtitles types are positioned so they don’t overlap.

Display Parameters

Subtitles are displayed using a TextBlock widget:

1. In Unreal Editor, go to Edit > Project Settings, scroll to Game, click on Subtitles and Closed Captions.

2. In the menu, use the dropdown to select DefaultSubtitleWidget.

3. Click the Browse to Asset icon to view the widget.

   

4. Double-click the widget to open it.

5. The default subtitle widget allows for independent display parameters (e.g. placement, font, color, opacity, etc.) for subtitles, closed captions, and audio descriptions. Double-click in the subtitles text box to open the Details tab.

   

Blueprint Access

In addition to USubtitleAssetUserData and the TextBlock widget parameters, the following API is exposed for use by C++ and Blueprints:

QueueSubtitle(const FQueueSubtitleParameters&, const ESubtitleTiming Timing = ESubtitleTiming::InternallyTimed)

Where FQueueSubtitleParameters contains:

1. The subtitle, a USubtitleAssetUserData, documented above.

2. An optional parameter to override the asset’s duration.

   If the subtitle is already being actively displayed, the active subtitle will have its duration updated to the specified duration.

Timing

Set to ESubtitleTiming::ExternallyTimed to control the queuing and expiration of a subtitle manually, instead of using the Subtitle’s duration.  Externally-timed subtitles will remain active indefinitely, until manually removed with StopSubtitle().  For sequencer scenarios involving time dilation or warping, it’s best to use ESubtitleTiming::ExternallyTimed and manually queue and stop subtitles.When queueing multiple subtitles, the most recent of the highest priority is displayed.

Displaying Dynamically Generated Subtitles

FText from arbitrary sources (e.g. dynamically replicated from a multiplayer server) can be displayed as subtitles. Example:

USubtitlesSubsystem* Subsystem = NewObject<USubtitlesSubsystem>(pWorld, NAME_None, RF_Transient);

check(Subsystem != nullptr);

USubtitleAssetUserData* Subtitle = NewObject<USubtitleAssetUserData>(GetTransientPackage(), NAME_None, RF_Transient);

check(Subtitle != nullptr);
Subtitle->Text = <arbitrary FText input>;

const UAssetUserData& AssetUserData = *CastChecked<const UAssetUserData>(Subtitle);

Sequencer

Drag your subtitle asset onto a subtitle track in the Sequencer. Subtitle properties can then be edited by right-clicking on the subtitle and selecting Properties. In the sequencer, subtitle duration is controlled by the selection range start and end rather than its duration property.

Localization

The subtitle system supports localization via its localizable FText property.  The FTexts are localized to the current language on the client using the localization system's hashkey implementation – localization occurs before and independent of the subtitle system.  See Localizing Content in Unreal Engine | Unreal Engine 5.5 Documentation | Epic Developer Community.

Migration Guide

The Migration Tool copies the subtitle data in a given DialogueWave and/or SoundWave into the new Subtitle asset type. It puts the new asset into the given DialogueWave and/or SoundWave's AssetUserData. Already-migrated subtitles are updated if the given DialogueWave and/or SoundWave is migrated again.  However, if the remigration results in no change, the target asset isn’t marked dirty and need not be resaved to disk.

Migration is non-destructive; old and new subtitles will be displayed automatically by default, letting the user control which content is used.

Display parameter settings (e.g. font, color, etc.) are not currently part of the migration tool.  These are set by category via the TextBlock widget settings documented above.

Migrating SoundWaves

To use the tool, select one or more SoundWave assets in the Content Browser. Then, right-click, find the Scripted Editor Actions submenu, and select Create Subtitle.  This will create new USubtitleAssetUserData to the SoundWaves that contain the migrated subtitle data.

When desired, one can remove data from the old subtitle system by:

1. Select one or more SoundWave assets in the Content Browser.

2. Right-click, find the Scripted Editor Actions submenu.

3. Select Remove Legacy Subtitles, which empties the old subtitle array...

   

Migrating DialogueWaves

After migrating the SoundWave associated with a DialogueWave, use the console command au.UseNewSubtitles 1 to see it with this subtitles plugin. You can switch back to using the original content and subtitle system by setting the console variable back to its default of 0.

Performance Notes

Disk usage varies slightly between the old and new system, with subtitles migrated to the new system requiring approximately 16 bytes more per subtitle (80 bytes in the old system, to 96 bytes in the new one) when queued for display.

Before (80 bytes per queued subtitle):

• FQueueSubtitleParams: 56 bytes

• FSubtitleCue: Multiples of 24 bytes. Assuming only one of these for this comparison.

After (96 bytes per queued subtitle):

• FQueueSubtitleParameters: 16 bytes

• USubtitleAssetUserData: 80 bytes

Unqueued subtitles sitting in memory do not use the FQueueSubtitleParameters structs, so "bulk" subtitles show a bigger difference at 24 bytes (old) to 80 bytes (new), with a difference of 56 bytes due to the additional properties and features.

Migration Tool Source

If modifications to the migration tool are necessary, it can be found in the subtitle plugin’s folder: SubtitlesAndClosedCaptions/Content/EditorUtilities/CreateSubtitlesFromSoundWaves.uasset

Like the rest of the new subtitle plugin, the migration tool is currently marked experimental.