Experimental Pixel Streaming Features

The below features are exciting new tools we've implemented into Pixel Streaming. Though they provide new possibilities, it's important to note that these are unstable and should be used with caution.

Pixel Streaming Player

The Pixel Streaming player allows you to display active pixel streams within your Unreal Engine project as a display within 3D space. This allows you to display cloud hosted content as a media source within local applications.

Setting up your Pixel Streaming Player

1. Enable the Pixel Streaming and Pixel Streaming Player plugins.
2. Create a new Blueprint class (actor). Save and name this anything you like.

3. Open the new blueprint class and add 2 components, PixelStreamingSignalling and PixelStreamingPeer.

   

4. Drag the PixelStreamingSignalling component into the event graph. From this node drag and create a "Connect" node. Connect BeginPlay to the new node and enter "ws://localhost" into the URL value.

   
5. Select the PixelStreamingSignalling component and add the; "On Config", "On Offer" and "On Ice Candidate" events from the details window. Add the "On Ice Candidate" event from the PixelStreamingPeer node as well.

1. From "On Config (PixelStreamingSignalling)" node, drag and create "Set Config (Pixel Streaming Peer)". Ensure the config values are connected between "Set Config" and "On Config".

   

2. From "On Offer (PixelStreamingSignalling)", drag off and create "Create Answer". Ensure the offer value is connected between "Offer" and "Create Answer". From the out node of "Create Answer", drag and create "Send Answer" node. Ensure Return Value of "Create Answer" connects to the answer value of "Send Answer".

   

3. From "Ice CandidatePixelStreamingSignalling)", Drag and create "Receive Ice Candidate" and connect the candidate values.

   

4. From "Ice Candidate(PixelStreamingPeer)", drag and create "Send Ice Candidate (PixelStreamingSignalling)" node. Connect candidate values.

   

5. If the above is done correctly, your finished blueprint should look like this:

   

6. Select the PixelStreamingPeer component in the blueprint. In the details window under Properties, you should see Pixel Streaming Video Sink. Select the drop down and choose "Pixel Streaming Media Texture". Name and save accordingly.

   
7. Drag your blueprint actor into the scene. Create a simple plane object and re-size and shape it into a suitable display.
8. Drag your saved Pixel Streaming media texture directly from the content browser onto the plane in the scene. This will automatically create a material and apply it to the object.
9. Start a basic local Pixel Stream external to this project. (Start signalling server and run application with relevant Pixel Streaming args)

10. Play your scene. You should now see your external Pixel Stream displayed on the plane in your scene!

    

VCam

VCam is a new feature that allows you to attach a VCam component to an in scene actor and stream the video content of the level viewport to an output provider.

At this stage, VCam is mostly intended for virtual production use cases. It can can be paired with the Live Link VCam iOS application and used for ARKit tracking. This is useful for piloting virtual cameras in Unreal Engine, with Pixel Streaming handling touch events and streaming the level viewport as a real time video feed back to the iOS device. For more information on Live Link VCam, please head to this site here: iOS Live Link VCam

How to use VCam

1. Ensure you have the Virtual Camera plugin enabled

   

2. Add the VCam component to any in scene actor (in the below example it has been attached to the Pixel Streaming player blueprint created in the guide above).

   

3. In the VCam component, navigate to "Output Providers" and add "Pixel Streaming Provider" in the drop down. Expand the new Output section.

   
4. To start and stop your stream, simply toggle the "Is Active" check box under Output.

5. Once started, open a local browser and navigate to "127.0.0.1" to see your streamed display, or open the Live Link iOS application and navigate to "127.0.0.1" and hit connect.

   

If you wish to interact with the stream through the browser, you will need to open the control panel in browser and change the Control Scheme to "Hovering"