Creating Virtual Keyboard Interactions

If you are looking to create a 3D keypad or virtual keyboard that allows a player to enter specified characters, you can do so with a combination of the Widget Interaction component and Widget components. The Widget Interaction component facilitates which "buttons" are clicked on and processes sending the characters through to a Widget component in an Actor that will handle the display.

In this how-to, we create a keypad that can be interacted with in and provides entries to a text box. As part of this how-to, we will determine which Widgets are focusable and which ones are not. This is important to the Text Box where the player is entering numbers does not lose player focus when a different button is pressed.

At the end of this guide, you will have a keypad and display that appears inside the 3D game world and can be interacted with.

For the purposes of this guide, we are using the Blueprint First Person template project.

1 - Character Setup

First, we need to set up our character with the Widget Interaction component and create the assets to display our UI.

1. Inside the Content > FirstPersonBP > Blueprints folder, open the FirstPersonCharacter Blueprint.

2. Add a WidgetInteraction component attached to the Sphere. In the Details panel, then zero out the Location and check Show Debug.

   

   The Widget Interaction component will now point in the direction that player points the gun, which is indicated in-game by a debug marker.

3. In the Event Graph, add a Right Mouse Button Event and drag in the Widget Interaction Component.

4. Off Widget Interaction add Press Pointer Key and Release Pointer Key nodes. Set both keys to Left Mouse Button. Connect the Pressed and Released pins from the Right Mouse Button to the respective Pointer Key nodes.

   

   Widgets respond to mouse/touch interaction via Left Mouse Button click or touch controls, and we can simulate this interaction using the Press Pointer Key function. In our example we use the Right Mouse Button to simulate a Left Mouse Button click; however, you can use a gamepad button press or even Motion Controller Trigger press to fire off the Press Pointer Key function.

5. In the Content Browser create two Blueprint Classes based on Actor named Keypad and KeypadDisplay. Then add two Widget Blueprints called KeypadWB and KeypadDisplayWB.

   

Now, our character can interact with Widgets in the world. Next, we will set up our interactive Widgets.

2 - Widget Blueprint Setup

In this step, we create the layout of our interactive UI Widgets and script what happens when clicking a button.

1. Open the KeypadDisplayWB Blueprint and delete the Canvas Panel. In the Hierarchy, add a Size Box with a TextBox.

   

2. For the Size Box, set the preview to Desired on Screen and override the Width and Height to 100x50.

   

3. For the Text Box, set the Hint Text to 1212 and adjust the Font Size to 28.

   

   In this example, the player must use the keypad to enter the text 1212 to signal a successful entry but you could use any value you wish.

4. Compile and Save the KeypadDisplayWB Blueprint.

5. Open the KeypadWB Blueprint and delete the Canvas Panel. Set up the Hierarchy with 3 Buttons each with the text 0, 1 and 2 respectively.

   

   You can also press F2 on each of the Buttons in the Hierarchy to change their names to reflect which button name.

6. Add a second Horizontal Box containing two Buttons with text in them, one set to Del the other set to Ent.

   

   The keypad will only have three numerical buttons and two others for deleting (Del) or committing entries (Ent).

7. Hold Ctrl and select each of your Buttons, and uncheck Is Focusable (set to False).

   

   In order for this setup to work, we need to ensure that no other widgets we interact with will take focus away from the Keypad Display widget. If we left these set to true, when we go to click on a button, the focus will be taken away from the Keypad Display and shift to the button. This prevents us from passing the key press and value over to the Keypad Display.

8. In the Event Graph, add OnClicked Events for each of the Buttons.

   

9. Create two Variables, a String type called CharToSend and a WidgetInteractionComponent called WidgetInteraction.

   

10. Hold Alt and drag in CharToSend for each OnClicked Event. Set the variable value to the button text values previously set: 0, 1 and *2.

    

11. Hold Ctrl and drag in WidgetInteraction. Drag off the pin and add a Send Key Char node. Connect it to a CharToSend variable and the OnClicked Events.

    

    We use the Send Key Char function because Text Boxes in Slate do not get Keys sent directly to them. For example, if we press the A key, the Slate function will not recognize the key press and will not send A through. Instead, Slate functions listen for Send Char so the lower level system operating code can determine if Shift was held down while pressing the A key, or if Caps Lock was on while pressing A.

12. Off the Del OnClicked Event, use the Press and Release Key and set the Key to Backspace. Connect a WidgetInteraction variable to the Press and Release Key target.

    

13. Create an Event Dispatcher called EnterPressed and call it from OnClicked(Ent).

    

14. Compile and Save the KeypadWB Blueprint.

Our Widget Blueprints are now set up to display our keypad and keypad display.

3 - Setting Up the Actors

Next, we need to set up the Keypad and Keypad Display actors before we begin scripting the Keypad actor.

1. Open the KeypadDisplay Blueprint and add a Widget component with class set to KeypadDisplayWB and check Draw at Desired Size.

   

   Draw at Desired Size is a quick way to cause the render target to automatically match the desired size inside the Widget Blueprint.

2. Compile and Save the KeypadDisplay Blueprint.

3. Open the Keypad Blueprint and add a Widget component with class set to KeypadWB and check Draw at Desired Size. Uncheck the Window Focusable option for this Widget Component.

   

   We don't want this component to take focus away from the Keypad Display widget, so this needs to be unchecked.

4. Add a WidgetInteraction component and uncheck Enable Hit Testing (set to false).

   

This Widget Interaction component means we don’t have to get a reference to the WidgetInteraction component in the player character. This enables us to disable Hit Testing, since we don't want to generate any hover effects.

WidgetInteraction components by default are set to Virtual User Index 0 when they are created and all of them communicate using the Virtual User Index ID. Since the WidgetInteraction component in the player character is also using Virtual User Index 0, this is the same as getting a reference to it.

4 - Scripting the Keypad Actor

In this final section, we add our Widget Blueprints as Widget Components to our Actors and define what happens when the player clicks on the Enter Button. We also set up a test in-game which prints Success to the screen if the code is correct or Wrong Code if the code is incorrect.

1. In the Keypad Blueprint Event Graph, off Event Begin Play add a Cast to KeypadWB node. Then, add a Widget component and drag off to add a Get User Widget Object. Connect the Return Value to the Cast Object pin.

   

2. Off the As Keypad WB pin, add a Set Widget Interaction node and add a Widget Interaction component and connect it to Set Widget Interaction.

3. Off the As Keypad WB pin, call the Assign Enter Pressed node, which will create an Enter Pressed Custom Event.

   

   Now when we select the Enter button, this custom event will be called. We can use to check if the correct code was entered.

4. Create a Keypad Display Object Reference Variable and call it TargetDisplay. Check Instance Editable.

   

5. Create a Text Variable called GoalText. Compile the Blueprint and set the Default Value to 1212.

   

6. Hold Ctrl and drag in TargetDisplay to get the Widget and User Widget Object. Then add a Cast to KeypadDisplayWB node and connect the Return Value to Object.

   

7. Off the As Keypad Display WB pin, get the Editable Text Box and GetText (Text Box). Off the Return Value, add and Equals node. Drag in Goal Text and connect it to the Equals node.

   

   If you renamed your Text Box, you will need to search for the new name to find the component.

8. Add a Branch with two Print Strings. Set the String text to Success (True) and the other to Wrong Code (False).

   

9. Compile and Save the Keypad Blueprint.

10. In the viewport, drag the Keypad and KeypadDisplay Blueprints into the level and rotate/position them as desired.

11. In the Details panel for the Keypad Actor, set the Target Display variable to the KeypadDisplay in the level.

    

12. Save the viewport and select Play to interact with the widgets.

End Result

Approach the keypad display, aim the pointer at the display, and Right-click to focus on it. You can now look at the buttons and Right-click on the buttons to issue the Send Key Char command and pass the value to the display. We are using the Right Mouse button to interact with the widgets in the level, but you can swap this to use gamepad or Motion Controller button presses as well.

While this example only prints to the screen if the correct code was entered or not, you can instead fire off some event that opens a door, spawns an enemy, or any other form of gameplay related event.