BuildGraph is a script-based build automation system that features graphs of building blocks common to Unreal Engine (UE) projects. BuildGraph integrates with UnrealBuildTool, AutomationTool, and the Editor. BuildGraph can also be extended and customized for your projects.
BuildGraph scripts are written in XML, specifying a graph of user-defined nodes with dependencies between them. Each node consists of tasks executed in sequence to produce some sort of output (for example, compile this project, then cook, run this test, and so on). When asked to build a target (that is, a node or named output) BuildGraph will execute all of the nodes in the graph required to make that happen.
Unlike other build tools, BuildGraph is designed as a hybrid between a makefile-like scripting language and a build farm configuration script. It enables annotations for the type of machine that steps are supposed to be executed on, providing a list of recipients for failure notifications if a step fails, and groups nodes that should only be executed after an explicit user trigger. It also tracks the creation of output files from task execution in a way that lends to graph execution being distributed across a farm of machines (with nodes running in parallel where possible) and intermediate artifacts being transferred to and from a network share automatically.
Epic uses BuildGraph to prepare the Unreal Engine binary release, package samples for the marketplace, and implement pipelines for our own games (among other things). Several example BuildGraph scripts are provided in the [Unreal Engine Root Directory]/Engine/Build/Graph/Examples directory, and the script for creating a binary Unreal Engine distribution can be found at [Unreal Engine Root Directory]/Engine/Build/InstalledEngineBuild.xml.
Opening a script with Visual Studio will use the schema located at [Unreal Engine Root Directory]/Engine/Build/Graph/Schema.xsd to provide rich tooltips, validation, and Intellisense features while editing.
Writing BuildGraph Scripts
If you want to learn how to write your own BuildGraph scripts, it helps to know the various parts that go into making a Graph. Graphs are created with the following elements:
-
Tasks: Actions that are executed as part of a build process (compiling, cooking, and so on).
-
Nodes: A named sequence of ordered tasks that are executed to produce outputs. Before they can be executed, nodes may depend on other nodes executing their tasks first.
-
Agents: A group of nodes that are executed on the same machine (if running as part of a build system). Agents have no effect when building locally.
-
Triggers: Container for groups that should only be executed after manual intervention.
-
Aggregates: Groups of nodes and named outputs that can be referred to with a single name.
Scripts typically make frequent use of properties for reusable or conditionally defined values. Properties are declared with the <Property> element, and are scoped to the point of their first declaration. Properties referenced with the $(Property Name) notation are valid within all attribute strings, and will be expanded when the script is read. Properties that can be supplied by the user on the command-line can be declared with the <Option> element, and environment variables can be imported into properties using the <EnvVar> element.
Any element may be conditionally defined via the "If" attribute. See below for the syntax of conditional expressions.
BuildGraph is typically used for packaged games, so filtering and file manipulation are natively supported. Any attribute that accepts a list of files may consist of a Perforce style wildcard (matching any number of ..., * and ? patterns in any location), a full path name, or a reference to a tagged collection of files. Attributes are denoted by prefixing a # character. Files may be added to a tag set using the <Tag> task, which also enables the performance of union and difference style operations. Each node can declare multiple outputs in the form of a list of named tags, which other nodes can then depend on.
Graphs may be executed in parallel as part of a build system. To do so, the initial graph configuration is generated by running with the -Export=<Filename> argument (producing a JSON file listing the nodes and dependencies to execute). Each participating agent should be synced to the same changelist, and AutomationTool (UAT) should be re-run with the appropriate -SingleNode=<Name> argument. Outputs from different nodes are transferred between agents via shared storage (typically, a network share) the path to which can be specified on the command line using the -SharedStorageDir=<Path> argument. Note that the allocation of machines (and coordination between them) is assumed to be managed by an external system.
The syntax of elements used to define BuildGraph constructs is listed in the following sections.
