Interacting with the Pixel Streaming System

When you have the Pixel Streaming system set up as described in the Quick Start guide you don't need to do anything special in the gameplay code of your Unreal Engine application to manage the system at runtime. However, the Pixel Streaming Plugin does offer some optional ways for you to control and interact with the Pixel Streaming system to achieve specific effects. This page describes these additional options.

Freeze Frame

Encoding every frame rendered by the Unreal Engine application into a media stream has some downsides: it takes resources on the computer running Unreal Engine, sending all of those frames over the network uses up bandwidth, and depending on the available bandwidth the encoding may degrade the quality of the rendered images.

To minimize the use of GPU resources and network bandwidth, you may at times want to temporarily disable the Pixel Streaming Plugin from encoding and sending every frame, showing connected clients a single static image instead. For example:

• If the client is not actively interacting with the application and nothing in the virtual world is moving, you may want your application to freeze on the last rendered frame until something changes.

• You may want to show clients an arbitrary image, such as a loading screen or a message caption.

  To allow this, the Pixel Streaming Plugin offers Blueprint nodes that you can use to pause and resume encoding. You'll find these nodes in the Pixel Streaming Freeze Frame category in the Blueprint Editor:

  

• When you want to replace the media stream with a static image, use the Freeze Frame node.

  

  This node accepts an optional reference to a Texture Asset. If you provide one, connected clients will see the Texture you specify in their player window. If you don't provide one, connected clients will see the last rendered frame generated by the Unreal Engine application at the time you called this node.

• When you want to resume streaming every frame, use the Unfreeze Frame node.

  

Freezing on a single frame or image texture does not affect input coming from the browser. By default, the player page will still forward keyboard and mouse events to the Unreal Engine while frozen.

Communicating from UE5 to the Player Page

You can make your Unreal Engine application emit custom events to all connected player HTML pages, which you can respond to in the player's JavaScript environment. This lets you change your web page UI in response to gameplay events.

To set this up:

1. In your Unreal Engine application, any time you want to emit an event to the player page, use the Pixel Streaming > Send Pixel Streaming Response node. Specify a custom string argument to the node to indicate to the player page what event has happened.

Click to enlarge image.

Or for a more extensive example (this one sends a response to the player when they hit the 3 key):

Click to enlarge image.

1. In the JavaScript of your player page, you'll need to write a custom event handler function that will be invoked each time the page receives a response event from the Unreal Engine application. It will be passed the original string argument that was sent by the Send Pixel Streaming Response node. For example:

       function myHandleResponseFunction(data) {
           console.warn("Response received!");
           switch (data) {
               case "MyCustomEvent":
                   ... // handle one type of event
               case "AnotherEvent":
                   ... // handle another event
           }
       }
   Copy full snippet

   function myHandleResponseFunction(data) { console.warn("Response received!"); switch (data) { case "MyCustomEvent": ... // handle one type of event case "AnotherEvent": ... // handle another event } }

2. Register your listener function by calling the addResponseEventListener function provided by app.js. You pass this function a unique name for your event listener, and your function. For example:

   addResponseEventListener("handle_responses", myHandleResponseFunction);
   Copy full snippet

   addResponseEventListener("handle_responses", myHandleResponseFunction);

3. If you ever need to remove your event listener, call removeResponseEventListener and pass the same name. For example:

   removeResponseEventListener("handle_responses");
   Copy full snippet

   removeResponseEventListener("handle_responses");

If you want to pass more complex data, you can format the string you pass to the Send Pixel Streaming Response node as JSON. For example:

Responding to Pixel Streaming Events

The Pixel Streaming system offers a way for your application's gameplay code to respond to selected events that occur over the lifetime of the Pixel Streaming session. These events come in two flavors: blueprint events and native C++ events:

Blueprint Events

These delegates can be accessed through the UE5's Blueprint Editor.

The following Blueprint events are only available to Pixel Streaming 2.

To respond to these events, you bind to them using the Pixel Streamer Delegates class. You'll typically set up this binding at the beginning of gameplay; for example, you might do this in response to the Event BeginPlay event. Once the binding is set up, your custom event will get triggered each time the bound event is triggered.

1. In the Blueprint Graph Editor, drag right from any node's execution pin and choose Pixel Streamer Delegates > Get Pixel Streamer Delegates.

   

2. Drag right from its Return Value and expand the Pixel Streamer Delegates category. You'll see a number of new options for binding and unbinding events.

   

3. Choose the Bind... option for the event you want to respond to, such as Bind Event to On All Connections Closed. You'll get a new node with an Event input. Connect its input execution pin to the output execution pin from Get Pixel Streamer Delegates.

   

4. If you already have a custom event that you want to trigger, connect the Output Delegate pin in its title bar to the Event input on the Bind node you just created. Otherwise, drag left from the Event input and choose Add Event > Add Custom Event to create a new custom event.

   

5. Connect your custom event to the Blueprint logic you want to run in response to the Pixel Streaming event.

For example, this implementation calls a custom function defined in the same Blueprint to reset the application to its initial state:

C++ Events

Developers can register their own handlers to these Pixel Streaming delegates. We recommend developers refer to the multi-cast delegate documentation for specific details on C++ delegate usage in Unreal Engine. A simple example of using the Pixel Streaming delegates is:

	#include "PixelStreamingDelegates.h"

	void FExample::OnStatChanged(FString PlayerId, FName StatName, float StatValue)
	{
	//TODO: Do something with the stat that changed.
	}

	void FExample::BindToDelegates()
	{
	// Bind to the OnStatChangedNative delegate.
	if(UPixelStreamingDelegates* Delegates = UPixelStreamingDelegates::Get())
	{
	Delegates->OnStatChangedNative.AddRaw(this, &FExample::OnStatChanged);
	}
	};
Copy full snippet

#include "PixelStreamingDelegates.h" void FExample::OnStatChanged(FString PlayerId, FName StatName, float StatValue) { //TODO: Do something with the stat that changed. } void FExample::BindToDelegates() { // Bind to the OnStatChangedNative delegate. if(UPixelStreamingDelegates* Delegates = UPixelStreamingDelegates::Get()) { Delegates->OnStatChangedNative.AddRaw(this, &FExample::OnStatChanged); } };