There are a collection of tools for automating the process of building Unreal Engine, including UnrealBuildTool and UnrealHeaderTool (among others), and this guide provides an overview of these build tools, including a brief overview of Unreal Engine's modular architecture.
Note: If you would like to debug UnrealBuildTool or AutomationTool on Mac OS X, please install MonoDevelop 4.2.2 and Mono 3.2.7.