Song Sync

The Song Sync (S-SYNC) device allows you to play songs from other sources and synchronize them with Patchwork devices. Working in Unreal Editor for Fortnite, you can sync a level sequence to the imported sound data. When you import MIDI files into the device, you can generate Patchwork note data output to control other audio or events!

Possible uses:

• Set up a virtual concert with pre-recorded audio, then use Patchwork to control the timing of audiovisual effects in sync with that audio.

• Build musical gameplay experiences ranging from rhythm action, like Fortnite Festival, to platformers where hazards and enemies move to the music, or racing games where the song you select changes the track layout, and more!

• Play a pre-recorded vocal track and mix live Patchwork audio with it.

The video below gives a preview of possibilities when using the Song Sync device:

Device Options

UEFN Editor Options

The options appear in the UEFN Editor only, and you can choose which assets should play in sync when this device is played.

Other Device Options

The options below can be modified in the UEFN editor, but you can also modify them in Create mode by walking up to the device and pressing E to reveal the device options.

The following video shows you the steps needed to play your custom audio using the Song Sync device:

Playback Delay

Playback Delay is essentially the concept of quantization in music. It is like snapping notes to the beat. If a note is played a little early or late, quantization moves it to the nearest beat or rhythm spot, making the timing more exact. It helps the music sound more on time and organized.

With Playback Delay set to None and Play Behavior set to Play from Start, pressing Play on the Song Sync device will result in the sample starting at the exact moment the button is pressed, regardless of the measure count on the global timeline:

With Playback Delay set to Next Beat and Play Behavior set to Control Tempo, pressing Play will only start the sample at the start of the next beat. The Control Tempo setting will restart the measure count:

Looping

Looping determines whether the sample and/or MIDI file loaded into your Song Sync device will restart from the beginning after it finishes playing.

With Looping set to On MIDI Length and Play Behavior set to Control Tempo, the length of the MIDI file loaded into the device will determine where the device loops. The Control Tempo setting will restart the measure count:

With Looping set to On Sequence Length and Play Behavior set to Control Tempo, the length of the Level Sequence synced to the device will determine where the device loops:

The following video illustrates how quantization and looping work in the context of a real musical sample:

MIDI Files

The Song Sync device allows you to import and play MIDI files as though you were using the Patchwork Note Sequencer. Using the Note Out cable, you can patch Song Sync to an Instrument Player or an Omega Synthesizer to play the MIDI notes using Patchwork! You can also patch Song Sync output to a Note Trigger to drive Events timed to your MIDI data.

Be aware that some Digital Audio Workstations (DAWs) do not add a Tempo Map when exporting MIDI files. A MIDI file without a Tempo Map cannot control the tempo of Patchwork devices.

The following video takes a deeper dive into using the Song Sync device with a MIDI file:

When a Song Sync device Play Behavior is set to Control Tempo, it can use a MIDI file tempo map to determine the Patchwork tempo. The minimum Patchwork tempo is 60 BPM and the maximum is 180 BPM. Tempo values outside of that range will be clamped.

MIDI files can also change the Patchwork key and mode. To do this, open your MIDI file editor and add a text event in the following format at the MIDI time where you’d like the change to happen:

• key KEY_NAME MODE_NAME

Sharps are denoted with #, and flats with b. Currently, only major and minor modes are supported. Some example text events:

• key F# minor

• key Bb major

Direct Event Binding System

Direct event binding allows devices to communicate directly, which makes your workflow more intuitive, and gives you more freedom to focus on your design ideas.

Below are the functions and events for this device:

Functions

Events

Using Song Sync in Verse

You can use the code below to control a Song Sync device in Verse. This code uses all features of the Song Sync device API. Modify it to fit the needs of your experience.

using { /Fortnite.com/Devices }
using { /Fortnite.com/Devices/Patchwork }
using { /Verse.org/Simulation }
using { /UnrealEngine.com/Temporary/Diagnostics }

# A Verse-authored creative device that can be placed in a level

song_sync_example := class(creative_device):

    @editable

    SongSync:song_sync_device = song_sync_device{}

    # Runs when the device is started in a running game

    OnBegin<override>()<suspends>:void=

       
        Sleep(5.0)

        SongSync.Disable()

        Print("Device Disabled")

        Sleep(5.0)

        SongSync.Enable()

        Print("Device Enabled")
Copy full snippet

using { /Fortnite.com/Devices } using { /Fortnite.com/Devices/Patchwork } using { /Verse.org/Simulation } using { /UnrealEngine.com/Temporary/Diagnostics } # A Verse-authored creative device that can be placed in a level song_sync_example := class(creative_device): @editable SongSync:song_sync_device = song_sync_device{} # Runs when the device is started in a running game OnBegin<override>()<suspends>:void= Sleep(5.0) SongSync.Disable() Print("Device Disabled") Sleep(5.0) SongSync.Enable() Print("Device Enabled")

To use this code in your UEFN experience, follow these steps.

1. Drag a drum sequencer device onto your island.

2. Create a new Verse device named song_sync_example. To learn how to create a new device in Verse, see Create Your Own Device Using Verse.

3. Open Verse Explorer and double-click song_sync_example.verse to open the script in Visual Studio Code.

4. Paste in the code above, compile, and drag the Verse-authored device onto your island.

5. Select your Verse device in the Outliner.

6. In the device’s Details panel, assign the object reference for the DrumSequencer to the drum sequencer device on your island. You can use the eyedropper to pick the device in the viewport, or use the dropdown and search for the device.

7. Save your project and click Launch Session.

Song Sequencer API

See the song_sync_device API Reference for more information on using the drum sequencer device in Verse.