LLM Game Mechanics with Verse

The prior page covered how an LLM character can be used to drive gameplay. This page takes things a step further by breaking down the Persona template’s Verse code and explaining how Verse interacts with the LLM to use the NPC as part of the game mechanic.

Using Libraries and Modules

Verse requires you to add certain modules and libraries to your script so it can access different code bases and APIs that control devices, AI, user interfaces (UI), and more. 

In this example, the following modules and libraries were used:

• using { /Fortnite.com/AI } - Basic AI functionality.

• using { /Verse.org/Simulation } - Basic Simulation functionality to run systems and reduce computational demand.

• using { /Verse.org/ScenGraph } - Access to SceneGraph API and functionality.

• using { /Fortnite.com/Playspaces } - Access to a Verse API module used to get an array of players or teams, and subscribe to player-added and player-removed events functionality.

• using { /UnrealEngine.com/Conversation } - Basic conversation functionality that works with AI.

• using { /Verse.org/AgentGroup } - Access to a library of AgentGroup functionality.

• using { /Verse.org/Chat } - Access to Chat functionality that provides a way for the player to interact with the LLM.

• using { /Fortnite.com/Devices } - Basic Fortnite devices API functionality.

• using { /Verse.org/Random } - Basic Random functionality. This is used to randomize the NPC’s budget and for prompt generation.andom 

using { /Fortnite.com/AI }
using { /Verse.org/Simulation }
using { /Verse.org/SceneGraph }
using { /Fortnite.com/Playspaces }
using { /UnrealEngine.com/Conversations }
using { /Verse.org/AgentGroup }
using { /Verse.org/Chat }
using { /Fortnite.com/Devices }
using { /Verse.org/Random }

Setting Up the Buyer Personas

To set up the buyer personas, start by initializing the Voice Channel and Team classes related to the Persona.

# Initializing Voice Channel and Team classes related to Persona
team_member_info := class(has_voice_member_info){}
team_agent_group := class(agent_group(team_member_info)) {}

Both classes use the enum Character := enum: to label the different buyer personas. These labels are used by other classes to correspond the persona labels to matching persona parameters the LLM uses when formulating dialog and decisions.

# Enum we will use to roll for a random Character for their vehicular needs in this game
Character := enum:
    FamilyPerson,
    HeavydutyWorker,
    SinglePerson,
    OutdoorsPerson,
    Collector

Driving Gameplay with Structured Output

Two structs contain variable parameters for how the LLM responds when interacting with the player under certain circumstances:

• The player suggests a vehicle and agrees to the price the LLM is willing to pay.

• The NPC is removed from the game if the player has dismissed the NPC.

The first Structured Output named, FinalOfferResponse, uses the ai_description to fill in the variable FinalOffer:logic by determining whether the price the player suggested was acceptable for the vehicle.

The FinalOfferResponse struct then uses the variable FinalPrice:int = 0 to fill in the ai_description by determining whether to accept the player’s vehicle suggestion and price based on whether both the vehicle and price meet the Buyer Persona’s needs and the predetermined amount vehicle that persona is willing to spend on a vehicle.

The last Structured Output named, KickResponse := struct, uses the ai_description to fill in the variable named Kicked:logic = false. The LLM uses the variable to create a goodbye message. KickResponse is then used by the KickFunction to remove the NPC from the game if the player tells the NPC to leave.

# Structs used for structured output. We will use these to carry variables from our structured outputs to the functions we binded to them
FinalOfferResponse := struct {
    @ai_description("Return true whenever you and the player have reached to an agreed price that you are both happy with.")
    FinalOffer:logic = false


    @ai_description("Return an int that represents the amount of money that you and the player have agreed upon.")
    FinalPrice:int = 0
}
KickResponse := struct {

Coding NPC Behavior

To use Structured Output as instructions to control the Verse-authored NPC behavior, a new class named persona_behavior is created. You can assign this class within a Character Definition or with an NPC Spawner device to guide the LLM interactions. In the Persona template, the persona_behavior class maps the enums (buyer persona labels) with their matching string and vehicle budgets (determined in a range in the form of tuple(int,int)).

# A Verse-authored NPC Behavior that can be used within an NPC Character Definition or an NPC Spawner device's NPC Behavior Script Override.
persona_behavior := class(npc_behavior):

The persona_behavior class uses a function named MakeMessage to create messages for the LLM at runtime using the mapped strings. The MakeMessage function also creates the name of the voice channel. 

    # Helper function to make strings into messages at runtime
    MakeMessage<localizes>(String:string):message = "{String}"

Two variables named var OptionAgentgroup and var OptionVoiceChannel store the group of players in the conversation and the channel they are a part of.

    # Optional variables that we use to store the group of players in the conversation and the channel they are part of
    var OptionAgentGroup:?agent_group(team_member_info) = false
    var OptionVoiceChannel:?voice_channel(team_member_info) = false

Two more variables named var FinalOfferCancelable and var KickCancelable are used to keep track of the Structured Output event bindings by referencing their returned cancelable events as optional cancelables. 

    # Devices and Verse classes we use to connect gameplay and UI
    @editable
    var Spawner:npc_spawner_device = npc_spawner_device{}
    var UIManager:ui_manager = ui_manager{}

Using Gameplay Variables

The persona_behavior class then uses gameplay variables to provide instructions to the LLM for the player interactions. The first is a variable named GreetingMessage. It’s a single request to talk that is passed to the LLM when it spawns. The second variable, JoinCharacter, is used to insert an empty line between messages when using the Join() function.

Together, these variables provide the guardrails for the LLM’s decisions by determining which buyer persona inhabits the NPC and using the Structured Output to choose how the buyer persona should speak.

    # -- Gameplay variables --
    # Message we pass as our prompt for our first interaction with the NPC
    GreetingMessage<localizes>:message = "You just entered the office. Say hello to the Car Salesman, and state that you are here to buy a car from them."
    # Message added between messages when using Join() to create our prompts.
    JoinCharacter<localizes>:message = "\n"

A second function named MakeBudgetPrompt reads Budget:int to communicate the amount of gold a particular buyer persona has for the LLM prompts. This prevents the LLM from generating its own budget with random numbers.

    # Helper function to create a message that includes our budget for our prompts.
    MakeBudgetPrompt<localizes>(Budget:int):message = "Your budget is {Budget} Gold."

The next series of gameplay variables provides parameters for awarding and penalizing players based on the LLM interactions. If the LLM rejects the player’s vehicle price or vehicle, then the variables var PenaltyScore and var ShouldApplyPenalty run. These variables deduct points from the player. 

If the LLM accepts the player’s offer, then the variables var AcceptedOffer and var ShouldAwardScore run. These variables add points to the player. The last variable, var ShouldDespawn despawns the NPC character and interacts with the player one last time in the context of the LLM’s decision to either reject or accept the player’s offer.

    # Gold we remove if the NPC leaves
    var PenaltyScore:int = -200
    # Vars that control what will happen when the NPC stops talking
    # Should the NPC despawn? Could be due to it being told to leave or a final offer has been reached
    var ShouldDespawn:logic = false
    # Should we give the Player Gold equal to the AcceptedOffer?
    var ShouldAwardScore:logic = false
    # Should we apply a penalty to the Player's score?
    var ShouldApplyPenalty:logic = false
    # Var we use to save the returned amount of Gold from the Structured Output

Mapping Buyer Persona Prompts

Two maps are created to assign a buyer persona string and vehicle budget to the randomly selected buyer persona type. 

The first map, named CharacterPromptMap, provides instructions for the LLM about vehicle needs and persona background information in a string to help the LLM build a backstory for the buyer persona. The second map, named CharacterBudgetMap, provides the relevant budgetary information for the randomly selected buyer persona.

message to the prompt.
    CharacterPromptMap:[Character]string = map {
        Character.FamilyPerson => "You are a Family Person who needs a vehicle to travel with their family. Generate a backstory and your needs for your vehicle based on this.",
        Character.HeavydutyWorker => "You are a Heavy-duty Worker who uses their vehicle for work. Generate a backstory and your needs for your vehicle based on this.",
        Character.SinglePerson => "You are a Single Person who uses their vehicle mostly for commuting and short-distance traveling. Generate a backstory and your needs for your vehicle based on this.",
        Character.OutdoorsPerson => "You are an Outdoors Person who goes on adventures often. Generate a backstory and your needs for your vehicle based on this.",
        Character.Collector => "You are a Car Collector who is looking for a one-of-a-kind vehicle. Generate a backstory and your needs for your vehicle based on this."
    }
    # Similarly to CharacterPromptMap, we use this to find the proper range based on the Character selected to randomize a budget for the client. These numbers are multiplied by 50 when using GetBudget().
    CharacterBudgetMap:[Character]tuple(int, int) = map {

Setting Up Personas

The OnBegin function runs when the NPC spawns into the game, the function’s main use is to randomly select one of the buyer personas and assign it to the NPC Spawner. The function is also responsible for:

• Subscribing to a number of events and Structured Output functions.

• Feeding a message to the caption UI.

• Providing a way for the LLM to react to the player’s input.

• Prompting the LLM into greeting the player when a new random character spawns on the NPC Spawner.

TIP: For an in-depth description of how Structured Output works in Verse and how to create gameplay with an LLM using Structured Output, see Create a Persona.

OnBegin finds and creates a reference to the UIManager so the conversation can pass information to the ui_manager device, such as the player’s score, triggering related UI animations, as well as providing the content for the captions. Everything related to the UI relies on the UIManager.

    # This function runs when the NPC is spawned in the world and ready to follow a behavior.
    OnBegin<override>()<suspends>:void=
        if:
            NPCEntity := GetEntity[]
            PersonaComponent := NPCEntity.GetComponent[persona_component]
            Player := NPCEntity.GetPlayspaceForEntity[].GetPlayers()[0]
        then:
            # Find all objects with the Tag 'ManagerTag' and check if they are a 'ui_manager'. If they are, keep a reference to it to send UI events/calls
            UIManagers := Self.FindCreativeObjectsWithTag(ManagerTag)
                for(Manager:UIManagers):

Cleaning Event Handlers

The OnEnd function runs to clean up existing event handlers if they have been assigned. 

    # This function runs when the NPC is despawned or eliminated from the world.
    OnEnd<override>():void=
        # It is good practice to clean up your cancelable events OnEnd(), so we are doing that here
        if(OfferEvent := FinalOfferCancelable?):
            OfferEvent.Cancel()
        if(KickEvent := KickCancelable?):
            KickEvent.Cancel()

Creating the Voice Channel

A channel for the conversation is created using the CreateAudioTeamAndAddPlayer function. The function gathers everything it needs for the channel: the NPC, player, and playspace. Before assigning a target of the conversation, the function places the NPC, player, and playspace into a voice_channel and names the channel TeamGroup. 

Next, the channel is configured as the place where all conversations (current and future) take place, and all collected game pieces (NPC, player, and playspace) for the conversation are added to the channel. 

    # Function that allows us to create the channel where the Player will converse with the NPC
    CreateAudioTeamAndAddPlayer(Player:player):void =
        if:
            NPCEntity := GetEntity[]
            NPCAgent := GetAgent[]
            SimEntity := NPCEntity.GetSimulationEntity[]
            PersonaComponent := NPCEntity.GetComponent[persona_component]
        then:
            # Create an 'agent_group' with our new agent_group class that uses team_member_info
            TeamGroup:agent_group(team_member_info) = agent_group(team_member_info){}

Targeting Players

The AddPlayerAsConversationTarget function is used in all LLM conversation scripts. The main duty of this function is to gather all the required pieces to set up the live conversation between the NPC and the player in the playspace. When the function successfully gets everything it needs to build the conversation, it registers the player as the conversation target for the parent NPC of this script.

    # Helper function that sets the passed Player as a valid conversation target for this behavior (if they are in the voice channel)
    AddPlayerAsConversationTarget(Player:player):void=
        if:
            NPCEntity := GetEntity[]
            NPCAgent := GetAgent[]
            TeamVoiceChannel := OptionVoiceChannel?
        then:
            if (PersonaComponent := NPCEntity.GetComponent[persona_component]):
                Player.SetConversationTarget(PersonaComponent, TeamVoiceChannel)

Selecting a Random Character

Buyer personas are selected at random using the ChooseCharacter function. All buyer persona enums are listed in the Character array. When one of the buyer personas occupies the SelectedCharacter designation, the appropriate buyer persona data maps to the SelectedCharacter using CharacterPromptMap and CharacterBudgetMap. 

When all the information is gathered and mapped, a new message is created and appended to the existing Personality. The LLM then uses this new Personality to greet the player and discuss the buyer’s vehicle needs and budget.

Mapping the buyer persona data provides an efficient way to pass the LLM all the information it needs to interact with the player without long delays.

    # Helper function to select a random character that we will use to modify our NPC's Personality
    ChooseCharacter():void=
        # Create an array of the possible Character enums that can be chosen
        Characters:[]Character = array{Character.FamilyPerson, Character.HeavydutyWorker, Character.SinglePerson, Character.OutdoorsPerson, Character.Collector}
        if:
            SelectedCharacter := Characters[GetRandomInt(0, Characters.Length - 1)]
            NPCEntity := GetEntity[]
            PersonaComponent := NPCEntity.GetComponent[persona_component]
            Budget := CharacterBudgetMap[SelectedCharacter]
            CharacterMapString := CharacterPromptMap[SelectedCharacter]

Initiating the Conversation

The InitialTalk function kicks off the conversation between the LLM and the player. It calls for the PromptToTalk() function using GreetingMessage as its message and uses the OptionVoiceChannel that was saved for talking to the player.

    # Function that prompts the NPC to talk to the Player. It uses our GreetingMessage as the prompt it should respond to
    InitialTalk(Persona:persona_component):void =
        if(Channel := OptionVoiceChannel?):
            option{Persona.PromptToTalk[GreetingMessage, Channel]}

Bartering with the LLM

The gameplay for this template centers around the player bartering with the LLM and trying to find a price the LLM is willing to accept for the vehicle. 

The driver of the bartering and gradual acceptance of the final price is the function named RegisterFinalOfferAction. The function gathers the information generated by the LLM to track the budget communicated to the player and uses the Structured Output to monitor the conversation about the vehicle’s price.

When the LLM accepts a final price for the vehicle, FinalOfferBinding then runs MakeMessage to create a success message. 

A failsafe is built into the interaction using the struct FinalOfferResponse. This is used to register the LLM’s actions and return the resulting cancelable success or failure check. After the RegisterFinalOfferAction() function succeeds, the LLM accepts the player’s offer, the acceptance is registered and the LLM prepares to run the success message.

    # It returns a '?cancelable' type that we can subscribe later on to keep track on the Structure Output
    RegisterFinalOfferAction():?cancelable =
        if:
            NPCEntity := GetEntity[]
            PersonaComponent := NPCEntity.GetComponent[persona_component]
            Session := PersonaComponent.GetAISession()
        then:
            # Define the binding we will use to determine when LLM should return a response through Structured Output and run the bound function
            FinalOfferBinding:prompt_binding_definition = prompt_binding_definition:
                Name:= MakeMessage("FinalOfferBinding")

Accepting the Offer

The gameplay comes to an end when the LLM agrees to pay the suggested amount for the vehicle. The FinalOfferFunction(Response:FinalOfferResponse) runs when the LLM has accepted the player’s offer through the Structured Output actions we registered.

The player receives gold from the NPC, and the final score is sent to the ui_manager device, and registers that the NPC should despawn from the playspace.

    # This is the function binded to our 'FinalOfferBinding'. It uses the 'FinalOfferResponse' to receive the parameters passed by the LLM
    FinalOfferFunction(Response:FinalOfferResponse):void=
        if(Response.FinalOffer?):
            # Award score to the player. Will be called in 'StopSayChecks'
            set ShouldAwardScore = true
            # Save our agreed upon price between the Player and the LLM. This will be used by our ui_manager
            set AcceptedOffer = Response.FinalPrice
            # Despawn the NPC. Will be called in 'StopSayChecks'
            set ShouldDespawn = true

Despawning NPCs

During the course of the game, the NPC Spawner device spawns random characters. Instead of disabling the device, the next two functions are used to either dismiss an NPC or kick an NPC from the game after accepting an offer, then cycle to a new character from the Character Definition.

The first function, RegisterKickAction handles dismissing the NPC when a player has asked the LLM to leave. This works similarly to RegisterFinalOfferAction. The function gathers information on the conversation session, then KickFunction is called.

    # Similar to the function 'RegisterFinalOfferAction', but this one handles whether or not the NPC was asked to leave
    RegisterKickAction():?cancelable =
        if:
            NPCEntity := GetEntity[]
            PersonaComponent := NPCEntity.GetComponent[persona_component]
            Session := PersonaComponent.GetAISession()
        then:
            KickBinding:prompt_binding_definition = prompt_binding_definition:
                Name:= MakeMessage("KickBinding")
                Description:= MakeMessage("Signals whenever you have been told to leave, you are told they will no longer deal with you, to look for business elsewhere, or any sentiment similar to those mentioned before.")

Next, KickFunction checks to see if the NPC has been kicked from the session. If it returns true, then a penalty is applied to the player and the NPC despawns.

    # Similar to 'FinalOfferFunction', but this will be called in response to the 'KickBinding' description being met
    KickFunction(Response:KickResponse):void=
        if(Response.Kicked?):
            # The NPC left, so give the Player a penalty. Will be called in 'StopSayChecks'
            set ShouldApplyPenalty = true
            # Despawn the NPC. Will be called in 'StopSayChecks'
            set ShouldDespawn = true

Both variables are checked in the StopSayChecks function to determine how to handle the parameters for deducting points from the player and despawning the NPC.

Controlling the Interactions

The StopSayChecks function fires every time the NPC finishes talking. This provides a space between the NPC interacting with the player and gameplay resulting in points being awarded or deducted and these results showing in the UI. 

The function checks ShouldApplyPenalty and ScoreChange to determine whether to proceed with PenaltyScore, AcceptedOffer, and DespawnNPC. When all three checks return true, then the variables PenaltyScore and AcceptedOffer apply their scores to the UI, and the DespawnNPC function runs resulting in the removal of the NPC.

    # This function fires every time the NPC has finished saying something to the Player. We call all of our gameplay functions here so that the NPC can finish speaking BEFORE doing anything gameplay related
    StopSayChecks(Interrupted:logic):void=
        # Handles penalties to the Player's score
        if(ShouldApplyPenalty?):
            # This function handles the UI changes for the score
            spawn{UIManager.ScoreChange(PenaltyScore)}
            # Reset the value for the next call
            set ShouldApplyPenalty = false
        if(ShouldAwardScore?):
            # This function handles the UI changes for the score

Passing Information to the UI

The persona_behavior not only controls the gameplay, it also passes score information to the ui_manager device through the four functions below.

The SetCloseCaption function checks for the latest interaction between the player and the LLM, then takes the LLM’s dialog and passes it to UIManager.

    # This function calls ShowCaptions() to make the Captions element visible and set the message inside it to the latest talk point from the NPC
    SetCloseCaptions(NPC:?agent, Message:message):void=
        UIManager.ShowCaptions(Message)

The ClearCloseCaption function determines when the LLM has stopped talking and clears the captions from the UI.

    # This function clears the contents of the Captions element and hides it when the NPC is done talking
    ClearCloseCaptions(Interrupted:logic):void=
        UIManager.HideCaptions()

The DespawnNPC function places a space between the existing NPC exiting the playspace and a new NPC who enters the playspace. This results in firing another function which plays a fade animation on the UI.

    # Helper function with a delay to match the Fade animation
    DespawnNPC()<suspends>:void=
        Sleep(0.8)
        Spawner.DespawnAll(false)

The GetBudget function uses Range:tuple(int, int) to determine a random budget amount for the NPC in the nearest multiple of 50.

    # Helper function to generate a random amount of gold and round it down to the nearest multiple of 50
    GetBudget(Range:tuple(int, int)):int =
        return Value:= GetRandomInt(Range) * 50

Complete Script

using { /Fortnite.com/AI }
using { /Verse.org/Simulation }
using { /Verse.org/SceneGraph }
using { /Fortnite.com/Playspaces }
using { /UnrealEngine.com/Conversations }
using { /Verse.org/AgentGroup }
using { /Verse.org/Chat }
using { /Fortnite.com/Devices }
using { /Verse.org/Random }
 

  Each page below is independent of the other pages. Click the topic you're interested in knowing more about.  

• 

  LLM Characters as Gameplay Drivers

  Drive gameplay using LLM characters.

• 

  LLM Caption UI

  Create a closed-caption user interface for an LLM dialog.