Audio Insights Plugin

Overview

AudioInsights is an engine plugin that provides a suite of tools to profile, debug, and monitor runtime aspects of audio in the Unreal Engine. Using several tabbed windows, AudioInsights can provide both visualizations and numerical values for current pitch, volume, and other parameter values for Sources, Audio Buses, Submixes, and more during live gameplay. It can provide information for PIE sessions in addition to standalone games.

Quick Start

AudioInsights is not enabled by default. To enable it, select Edit > Plugins in the Unreal Editor and enable the AudioInsights plugin. You will need to click Restart Now for the plugin to be loaded.

Once enabled for the project, you can open an AudioInsights window from the Tools menu in the editor window, select Audio Insights:

Beginning a PIE session will start populating the AudioInsights window with data from the audio engine. AudioInsights requires trace recording using UnrealInsights, and opening the AudioInsights window will automatically start recording a trace when you enter a PIE session. AudioInsights is a tabbed window and can be docked anywhere in your editor for your convenience. 

Using AudioInsights with a standalone game requires the use of UnrealInsights. From the Trace menu in the lower right side of the editor window, select Unreal Insights (Session Browser), then exit the editor when UnrealInsights has started.

Alternatively you can run UnrealInsights from the command line with:

/Engine/Binaries/Win64/UnrealInsights.exe -DisableFramerateThrottle

The extra flag will enable UnrealInsights to run at 60fps even when not in focus. Start your standalone game with the -Messaging flag and connect to it from UnrealInsights by using the Connection tab, ensuring cpu, audio, and audiomixer (with no spaces after the commas) are indicated in the Initial channels field, then click Connect.

This will connect to the currently running game. In the Trace Store tab, find the trace marked LIVE and double click to bring up the trace session window. In the Menu, select the Audio Insights option.

This will bring up an AudioInsights window that works similar to the one connected to the PIE above. A standalone instance of AudioInsights does not have the same messaging as working with PIE, only the Sounds, Sources, and Virtual Loops tabs are available.

Audio Data Detail Tabs

Visible tabs are configured using the View menu in the Audio Insights tab:

Detailed tabs can be repositioned anywhere within the Audio Insights tab or pulled off as a free floating window. Selecting Reset Layout will fix any uncontrolled tab chaos that results.

Top Level Controls:

• Only trace audio signals during PIE: AudioInsights uses the UnrealInsights tracing capabilities which can generate a lot of data. If you are only enabling the trace for AudioInsights, enabling this checkbox will reduce the amount of data stored as you monitor and debug your session with AudioInsights

• World Filter: this pull down menu gives you the option to select audio data from the currently running worlds, including the editor

Log Tab

The Log tab is another convenient way to filter and observe Unreal Editor logs next to any pertinent audio information you may be looking at. This data defaults to displaying all audio specific log categories, but you can change the filters to include or exclude any other log categories and verbosities.

Sounds Tab

The Sounds tab displays all the currently playing sound assets, listed by category, with text displays of runtime data for each. It enables you to interact with the playing sounds to mute and solo elements of the mix. Names used to identify sounds are the Uasset name and not the runtime instance name.

Sound Categories

• MetaSound: MetaSound Source assets

• Sound Cue: Sound Cues and any child Sound Waves

• Procedural Source: Any mixer input object that inherits from USoundWaveProcedural in code

• Sound Wave: simple wave asset

• Sound Cue Template: Sound Cues built at runtime from a template

• Other: Audio Source Buses

Data

• Muted: displays if this instance of the sound is muted

• Soloed: displays if this instance of the sound is soloed

• Name: UAsset name of the sound, not the runtime instance name

• Priority: set for concurrency, a positive float value with higher numbers indicating greater priority

• Distance: positive distance from sound renderer in map units

• Amp (Peak): value of an envelope follower tracking the sound amplitude, float value from 0.0 to 1.0 for maximum output amplitude

• Volume: the sum of all volume values and modulators for this instance of the sound excluding distance attenuation, a positive float value as a linear amplitude multiplier

• Pitch: the sum of all pitch values and modulators for this instance of the sound, a positive float value as a linear playback rate multiplier

• Aggregation: in a hierarchical object such as a Sound Cue that contains several child Sound Waves, the Sound Cue Volume and Pitch will show the top level values, but the Sound Wave will show the aggregated value of the Sound Cue top level and any other modulations that happen to that Sound Wave within the Sound Cue

Display Options

• Show Stopped: when selected, this will enable sounds that have recently stopped to continue to show in the list grayed out for a couple seconds, making it easier to observe short sounds

  

• Filter Categories: show category buttons in the tab header allowing quick filtering of sounds listed

  

• Data Display: right-clicking on the header bar will give options to select audio data displayed

  

Controls

• Sound Context Menu: right-clicking on a selected sound will allow you to Pin a sound to the top of the display, Browse to Asset in the content browser, or Edit to open up the asset editor from this window

  

• Mute Selected: while a sound is selected in the list, pressing the Mute Selected button will mute that sound asset

  

• Solo Selected: while a sound is selected in the list, pressing the Solo Selected button will solo that sound asset

  

• Clear All Mutes/Solos: resets to the default playing mix

Sources Tab

The Sources tab displays all the current inputs to the Audio Mixer in a flat list. The assets presented are consistent with the assets listed in the Sounds tab, displaying different audio data.

Displayed Data

• Play Order: monotonically increasing sound instance ID, indicates the order in which sounds were started in the mixer

• Name: Uasset name, not runtime instance name

• Amp (Peak): value of an envelope follower tracking the mixer source amplitude

• Volume: sum of all the volume controls and modulators for this mixer source not including distance attenuation, positive float value as a linear amplitude multiplier

• Distance Attenuation: gain multiplier based on distance from this mixer source to the sound renderer, float value of 0.0 to 1.0

• Pitch: sum of all the pitch controls and modulators for this mixer source, positive float value as a linear playback rate multiplier

• LPF: sum of all low pass filter controls and modulators for this mixer source indicating the cutoff frequency in Hertz

• HPF: sum of all high pass filter controls and modulators for this mixer source indicating the cutoff frequency in Hertz

Controls

• Data Filter: right-clicking on the list header bar will reveal a context menu to select which audio data values are shown per mixer source

  

• Global Mute/Solo: this control depends upon what is being filtered by the text window, the M and S buttons will mute and solo the filtered list of sources. If nothing is being filtered, pressing the M and S buttons will return to the default mix. This pre-empts any mute and solo actions on the Sounds tab

• Show/Hide Plot: pressing the plots button on the top right of the tab will show a time plot of selected data from the table

  

• Edit Asset: double-clicking on a mixer source in the list display will open the asset editor with that asset

Plot Trace

If Plot Trace is selected, the lower part of the Sources tab will show a line graph of the mixer source value histories for the chosen data for the most recent 5 seconds of time. Timeline values are from the start of the trace, typically the start of the gameplay session. The Plot data value is chosen using the pull down menu on the top right of the plot and displays in the same units as the numerical display described above. The Plot Trace will show the first 16 sources in the list.

Virtual Loops Tab

This tab displays a list of all Active Sounds that are currently being tracked by the renderer but not producing samples for the mixer due to concurrency or distance attenuation factors.

Displayed Data

• Play Order:  monotonically increasing sound instance ID, indicates the order in which sounds were started in the mixer

• Name: Uasset name for the virtualized mixer source

• Time (Virtualized): the time in seconds since this sound last sourced samples for the mixer

• Time (Total): the total time in seconds this sound has been playing, both virtually and not

• Update Interval: the time in seconds between position or concurrency updates that would determine if this sound remains virtual. This value varies with distance from the renderer

Controls

• Browse: this button will show the selected asset in the Content Browser

• Open: this button will open the asset editor for the selected asset

• Data Filter: right clicking on the list header bar will reveal a context menu to select which audio data values are shown per mixer source

  

• Edit Asset: double-clicking on a mixer source in the list display will open the asset editor with that asset

Submixes Tab

This tab lists all the Sound Submixes in the current project and indicates which ones have active signals at the moment. You can select a set of submixes to display signal levels in the Audio Meters tab.

Data Displayed

• Active: a dot in this column indicates an active signal on that submix

• Select: a checkbox in this column indicates the submix levels are being shown visually on meters in the Audio Meters tab. Note, you need to manually open the Audio Meters tab if it is not shown in your current configuration

• Name: the Sound Submix asset name

Controls

• Edit Submix: double-clicking a submix in the list will open the asset editor for that asset

Audio Buses Tab

Audio Buses are buffers of a given channel configuration to which any audio source can "send" audio. Audio sent to a bus is mixed together into a single signal. Audio buses are not themselves connected in any sort of graph or relationship to each other. Their utility is that they are a place to perform audio analysis (e.g. envelope following) that can drive other systems or they can be routed to "source buses" (which sonify their output to a sound source) or can be read from a MetaSound (and sonified or analyzed further in a MetaSound DSP graph).

Data Displayed

• Active: a dot in this column indicates an active signal on that Audio Bus

• Select: a checkbox in this column indicates the Audio Bus levels are being shown visually on meters in the Audio Meters tab. Note, you need to manually open the Audio Meters tab if it is not shown in your current configuration

• Name: the Audio Bus asset name

Controls

• Type Filter: enables you to display Audio Bus assets, buses generated in code, or both

  

• Edit Audio Bus: double-clicking a Audio Bus in the list will open the asset editor for that asset

Audio Meters Tab

This tab shows metered visualizations of both Submixes (green text) and Audio Buses (blue text) selected on each of those tabs. The metered amplitude uses a logarithmic dB scale. Each channel in a bus configuration is represented with a separate meter in canonical speaker orders. The window will scroll to accommodate all selected submixes and buses. There are no controls for this tab.

Analyzers Tab

The Analyzers tab is a series of 5 realtime visualizations of the engine’s main audio output. Select the visualizations from the ellipses menu on the upper right of the tab.

Meter: Displays the main output RMS amplitude in a logarithmic dB scale. The number of channels matches the channel configuration of the audio device. There are no additional controls for this analyzer.

Oscilloscope: Displays a time domain sample amplitude plot of the most recent 0.5 seconds of audio coming from the main audio output. The scale is a linear float from 0.0 to a max of 1.0. There are no additional controls for this analyzer.

Vectorscope: Displays a linear time domain amplitude of the left channel on the horizontal axis against the right channel on the vertical axis. This is good for observing phase relationships and channel coherency. There are no additional controls for this analyzer.

Spectrogram: Displays a frequency domain plot of the most recent audio output mixed into a mono channel. Time is on the horizontal axis and audio frequency is on the vertical. Each vertical line in the plot represents a single timed window of audio and the color of the pixel represents the power of the audio signal at that frequency. There are several controls for the plot, selected by right-clicking anywhere on the analyzer:

• Analyzer Type: select between a Fast Fourier Transform (FFT) and a Constant Q Transform (CQT) to represent the audio signal in the frequency domain

• FFT Size: the number of audio samples used in each windowed transformation. A larger size will give greater frequency resolution, at the trade off of lower time resolution

• Pixel Plot Mode: choice of Sample, Peak, or Average in how the samples are weighted to display signal strength. This can be used to emphasize transient frequencies or not

• Frequency Scale: choice of linear or logarithmic scale to see different details in the spectrum

• Color Map: choice of white or black representing a greater signal strength

• Orientation: choice of horizontal (pictured above) or vertical which swaps the axes so time is vertical and frequency is horizontal

Spectrum Analyzer: Displays an instantaneous frequency domain plot of the main output mixed into a mono channel with audio frequency on the horizontal axis and signal strength on the vertical axis. Moving the mouse to a specific frequency will display the signal strength at that band of the spectrum. There are several display options for this analyzer.

• Ballistics: choice of Analog or Digital, which simulates the signal persistence between audio windows to have the appearance of analog or digital professional audio meters. This determines how smooth (analog) or precise (digital) the transition between frames can be

• Analyzer Type: select between a Fast Fourier Transform (FFT) and a Constant Q Transform (CQT) to represent the audio signal in the frequency domain

• FFT Size: the number of audio samples used in each windowed transformation. A larger size will give greater frequency resolution, at the trade off of lower time resolution

• Tilt Spectrum: an amount of attenuation given per frequency to flatten out the graph of a spectrum that is more heavily weighted on the lower frequencies

• Pixel Plot Mode: choice of Sample, Peak, or Average in how the samples are weighted to display signal strength. This can be used to emphasize transient frequencies or not

• Frequency Scale: choice of linear or logarithmic scale to see different details in the spectrum

• Display Frequency Axis Labels: labels the horizontal graph grid lines with numerical values in Hertz

• Display Sound Level Axis Labels: labels the vertical graph grid lines with numerical values in decibels

Control Buses Tab

The Audio Modulation plugin enables the user to create Control Buses that can modify pitch, volume, filter, and other values for audio sources and submixes. This tab displays the current value for each active bus in a numerical form, formatted as a float value between 0.0 and 1.0. The pitch, volume, and filter values seen on the Sources and Sounds pages reflect the influence of the Control Buses displayed here.

There are no additional display or control options on this tab.

Modulation Matrix

Control Buses in the Audio Modulation plugin can add together to create cumulative effects. The Modulation Matrix tab gives insight on the different buses currently active and how they are collaborating to affect audio sources.

  There are no additional display or control options on this tab.