Pixel Streaming Sample Project

The Pixel Streaming Demo showcase demonstrates how you can design Unreal Engine content for people to experience as a live stream, using a Web browser on either a desktop or mobile device. It includes:

• An HTML player page that plays back a media stream generated by your Unreal Engine application, and that offers custom UI elements for controlling the Engine remotely.

• An Unreal Engine Project that is already set up to generate a media stream using the Pixel Streaming Plugin, and to respond to the custom commands issued from the HTML player page.

You can use this sample as a model for building your own custom HTML5 player that interacts with your Unreal Engine content.

Prerequisites:

• Make sure that you understand the basics of the Pixel Streaming system.

• Follow all the instructions in the Getting Started guide at least once to make sure that you have everything that you need installed and working with the default player page.

Getting Started

1. Find the Pixel Streaming showcase on the Learn tab of the Epic Games Launcher, and use it to start a new Unreal Engine Project.

2. In Windows Explorer, browse to the location of the Project on your computer, and find the WebInterface folder.

3. Copy the contents of this folder to the Samples/PixelStreaming/WebServers/SignallingWebServer/custom_html folder under your Unreal Engine installation folder. 
   If that custom_html folder doesn't yet exist, create it first.

4. Open the PixelStreamingDemo.uproject file in the Unreal Editor.

5. Follow the process described in the Getting Started with Pixel Streaming page to:

   • Package the Project or start it from the Unreal Editor as Standalone Game.

   • Start the Signaling and Web Server.

6. Open a Web browser, and navigate to the PixelDemo.htm player page being hosted by your Signaling and Web Server. For example:
   http://<your-server-ip-address>/PixelDemo.htm

Controlling the Player Page

When you have the Pixel Streaming system set up correctly, and you use a supported Web browser to access the custom PixelDemo.htm player page, you can use the controls described in the following sections to interact with the running Unreal Engine application.

1. Use the menu on the left hand side of the page to change the current character, skin, environment, and time of day.

2. The Toolbar provides some additional controls over the UI and the viewer:

   Control	Effect
   	Shows and hides the left-hand menu.
   	Toggles the camera between its fully zoomed-in and fully zoomed-out positions.
   720p, 1080p, 4k	Sets the rendering resolution of the Unreal Engine application running on the server.

   Not all GPUs are capable of encoding video feeds at 4K resolution.

   |

   Shows and hides the Bandwidth Cap and Framerate Cap settings.

   Bandwidth Cap

   Sets the maximum bit rate of the encoding produced by the Pixel Streaming Plugin running on the server. As you lower this value, you'll start to see compression artifacts begin to appear in the rendered images. At the same time, as the compression reduces the bandwidth needed by the stream, you should see the rendered content in your player widget perform more smoothly over slower network connections.

   Due to a current limitation, once you cap the bandwidth to a given value, you will not be able to raise that limit again until you restart your Unreal Engine application.

   |

   Framerate Cap

   Sets the maximum frames the Unreal Engine application will generate per second. This value is limited to 60 fps by default. Removing this cap may lead to hitches and stutters if the frame rate generated by the Unreal Engine exceeds the capacity of the network stream to transmit the images.

3. The viewer widget itself offers direct mouse and touch control over the Unreal Engine application:

   Control	Effect
   Click and drag, or touch and drag	Rotates the camera around the current character.
   Mouse wheel	Zooms the camera in and out.
   	Switches the viewer to full-screen mode. Press the Esc key to exit.

4. Click the Add (+) button at the top right of the page to toggle an overlay that provides additional controls over the stream. Most of these settings are the same as the ones in the default player.htm page, and are described in the Getting Started guide. The new settings are:

   Control	Effect
   Prioritise Quality	When active, the rendering resolution of the Unreal Engine application is automatically adjusted to match the current size of the player widget in the HTML player page. As you make the browser window smaller, and the size of the player widget decreases, the rendering resolution of the Unreal Engine application on the server decreases to match. Similarly, if you enter full-screen mode on the player page, the Unreal Engine application scales up its window to match the resolution of the client's screen.
   Match Viewport Resolution	When active, the Pixel Streaming system automatically limits the number of frames per second produced by the Unreal Engine application when the available bandwidth for the connection is limited. See the descriptions for the Streamer.PrioritiseQuality, Streamer.LowBitrate, Streamer.HighBitrate and Streamer.MinFPS console commands in the Pixel Streaming Reference.

Understanding the Custom UI Events

The custom HTML player page, PixelDemo.htm, uses the scripts/app.js file provided with the Pixel Streaming Signaling and Web Server to capture mouse, keyboard, and touch events and relay them back to the Unreal Engine application. In addition, it relies on its own custom PixelDemo.js file for some additional functionality.

Most of the UI elements in the player page are implemented by calling the emitUIInteraction() function to pass different information back to the Unreal Engine application. To understand how any piece of the UI works under the hood:

1. First, find the UI item you want to understand in the PixelDemo.htm file, and see what JavaScript function is set to trigger when that button is pressed.
   For example, each item in the left-hand menu triggers a function called onConfigButton(), and passes it two numbers.

2. Fortress

3. Look for the definition of that function in the PixelDemo.js file.
   For example, the onConfigButton() function writes these two numbers into a JavaScript object as the values of the Category and Item fields respectively, then it passes that JavaScript object to the emitUIInteraction() function:

   function onConfigButton(category, item) { let descriptor = { Category: category, Item: item }; emitUIInteraction(descriptor); console.log(descriptor); }

4. In the Unreal Engine Project, these events are responded to by the Blueprints/Pawn/Orbit_Controller class.

   

   Double-click this class to open up its Event Graph.

5. In the Bind JSON Events area, you'll see how the Bind Event to OnPixelStreamingInputEvent node is used to register the OnJSONEvent event as a handler each time emitUIInteraction() is called in a connected browser.

6. Each time the OnJSONEvent event triggers, the Blueprint calls Get Json String Value to check the keys and values stored in the JavaScript object that was passed by the player page to the emitUIInteraction() function. It uses those values to decide what other events it should trigger.
   For example, when the event receives an input that contains a Category field, it calls the Change Config Option event.

   This event in turn checks the value of the Category field, and uses that value to determine whether it should change the character, the skin, the environment, or the time of day.

   Each of these events in turn checks the Item value to decide what type of character, skin, environment, or time of day it should activate.

The controls in the overlay at the top right of the page, such as Prioritise Quality and Match Viewport Resolution, work a slightly different way. The app.js file adds event listeners to these HTML elements when it is loaded. Each time the user enables or disables one of these elements, the listener function in the app.js file is triggered to respond directly to the change.