1. Epic GamesDeveloper
  3. FAutomationTestFramework

FAutomationTestFramework

Class representing the main framework for running automation tests

API > API/Runtime > API/Runtime/Core

Class representing the main framework for running automation tests

   
Name FAutomationTestFramework
Type class
Header File /Engine/Source/Runtime/Core/Public/Misc/AutomationTest.h
Include Path #include "Misc/AutomationTest.h"

Syntax

class FAutomationTestFramework

Constructors

Name Remarks Include Path Unreal Specifiers
Copy constructor and assignment operator intentionally left unimplemented. Misc/AutomationTest.h  
Constructor Misc/AutomationTest.h  

Destructors

Name Remarks Include Path Unreal Specifiers

~FAutomationTestFramework()

 Destructor Misc/AutomationTest.h  

Classes

Name Remarks
FAutomationTestMessageFilter Special feedback context used during automated testing to filter messages that happen during tests
FAutomationTestOutputDevice Special output device used during automation testing to gather messages that happen during tests

Variables

Public

Name Type Remarks Include Path Unreal Specifiers
OnAfterAllTestsEvent FSimpleMulticastDelegate Called after all chosen tests run have finished. Misc/AutomationTest.h  
OnBeforeAllTestsEvent FSimpleMulticastDelegate Called before all chosen tests run. Misc/AutomationTest.h  
OnPerformanceDataRetrieved FOnPerformanceDataRetrieved Called when the performance data is retrieved. Misc/AutomationTest.h  
OnScreenshotCompared FOnTestScreenshotComparisonComplete Called when a screenshot comparison completes. Misc/AutomationTest.h  
OnScreenshotComparisonReport FOnTestScreenshotComparisonReport Called when a screenshot comparison result is reported Misc/AutomationTest.h  
OnScreenshotTakenAndCompared FSimpleMulticastDelegate The final call related to screenshots, after they've been taken, and after they've been compared (or not if automation isn't running). Misc/AutomationTest.h  
OnTestDataRetrieved FOnTestDataRetrieved Called when the test data is retrieved. Misc/AutomationTest.h  
OnTestEndEvent FOnTestEvent Called when each automated test is ending Misc/AutomationTest.h  
OnTestStartEvent FOnTestEvent Called when each automated test is starting Misc/AutomationTest.h  
PostTestingEvent FSimpleMulticastDelegate Called after all automated tests have completed Misc/AutomationTest.h  
PreTestingEvent FSimpleMulticastDelegate Called right before automated test is about to begin Misc/AutomationTest.h  

Protected

Name Type Remarks Include Path Unreal Specifiers
AllExistingTags TSet< FString > Set of all tags whether they are code declared or through config file Misc/AutomationTest.h  
AutomationTestClassNameToInstanceMap TMap< FString, FAutomationTestBase * > Mapping of automation test names to their respective object instances Misc/AutomationTest.h  
AutomationTestMessageFilter FAutomationTestMessageFilter Specialized feedback context used for message filtering during automated testing Misc/AutomationTest.h  
AutomationTestOutputDevice FAutomationTestOutputDevice Specialized output device used for automation testing Misc/AutomationTest.h  
bCaptureStack bool   Misc/AutomationTest.h  
bDeveloperDirectoryIncluded bool Whether we want to run automation tests on content within the Developer Directories Misc/AutomationTest.h  
bForceSmokeTests bool Forces running smoke tests Misc/AutomationTest.h  
bTestSuccessful bool True if the execution of the test (but possibly not the latent actions) were successful Misc/AutomationTest.h  
CurrentTest FAutomationTestBase * Pointer to the current test being run Misc/AutomationTest.h  
CurrentTestFullPath FString Full test path as given by the automation controller of the active test Misc/AutomationTest.h  
ImmutableTags TMap< FString, TSet< FString > > Map of immutable tags for each test. Code defined tags are immutable. Misc/AutomationTest.h  
LatentCommands TQueue< TSharedPtr< IAutomationLatentCommand > > Queue of deferred commands Misc/AutomationTest.h  
NetworkCommands TQueue< TSharedPtr< IAutomationNetworkCommand > > Queue of deferred commands Misc/AutomationTest.h  
NetworkRoleIndex uint32 Participation role as given by the automation controller Misc/AutomationTest.h  
OnEnteringTestSectionEvent TMap< FString, FOnTestSectionEvent >   Misc/AutomationTest.h  
OnLeavingTestSectionEvent TMap< FString, FOnTestSectionEvent >   Misc/AutomationTest.h  
OriginalGWarn FFeedbackContext *   Misc/AutomationTest.h  
Parameters FString Copy of the parameters for the active test Misc/AutomationTest.h  
RequestedTestFilter EAutomationTestFlags Whether we are currently executing smoke tests for startup/commandlet to minimize log spam Misc/AutomationTest.h  
StartTime double Time when the test began executing Misc/AutomationTest.h  
TagFilter TSharedPtr< FTextFilterExpressionEvaluator > Cached filter context Misc/AutomationTest.h  
TestFullNameToTagDataMap TMap< FString, FString > Mapping of full test names to their registered tags. Misc/AutomationTest.h  
TestScreenshotAndTraceCapturedDelegate FOnTestScreenshotAndTraceCaptured Delegate called at the end of the frame when a screenshot and frame trace is captured and a .png is requested Misc/AutomationTest.h  
TestScreenshotCapturedDelegate FOnTestScreenshotCaptured Delegate called at the end of the frame when a screenshot is captured and a .png is requested Misc/AutomationTest.h  

Functions

Public

Name Remarks Include Path Unreal Specifiers

void AddAnalyticsItemToCurrentTest ( const FString& AnalyticsItem
)

 Adds a analytics string to the current test to be parsed later. Misc/AutomationTest.h  

bool CanRunTestInEnvironment ( const FString& InTestToRun,
FString* OutReason,
bool* OutWarn
) const

 Internal helper method designed to check if the given test is able to run in the current environment. Misc/AutomationTest.h  

bool ContainsTest ( const FString& InTestName
) const

 Checks if a provided test is contained within the framework. Misc/AutomationTest.h  

void DequeueAllCommands()

 Dequeue all latent and network commands Misc/AutomationTest.h  

void EnqueueLatentCommand ( TSharedPtr< IAutomationLatentCommand > NewCommand
)

 Enqueues a latent command for execution on a subsequent frame Misc/AutomationTest.h  

void EnqueueNetworkCommand ( TSharedPtr< IAutomationNetworkCommand > NewCommand
)

 Enqueues a network command for execution in accordance with this workers role Misc/AutomationTest.h  

bool ExecuteLatentCommands()

 Execute all latent functions that complete during update Misc/AutomationTest.h  

bool ExecuteNetworkCommands()

 Execute the next network command if you match the role, otherwise just dequeue Misc/AutomationTest.h  

TSet< FString > GetAllExistingTags()

   Misc/AutomationTest.h  

bool GetCaptureStack()

   Misc/AutomationTest.h  

FAutomationTestBase * GetCurrentTest()

 Returns the actively executing test or null if there isn't one Misc/AutomationTest.h  

FString GetCurrentTestFullPath()

 Returns the actively executing test full path Misc/AutomationTest.h  

FOnTestSectionEvent & GetOnEnteringTestSection ( const FString& Section
)

 Called entering test section. Misc/AutomationTest.h  

FOnTestSectionEvent & GetOnLeavingTestSection ( const FString& Section
)

 Called leaving test section. Misc/AutomationTest.h  

void GetPossibleRestrictedPaths ( const FString& BasePath,
const TArray< FString >& RestrictedFolderNames,
TArray< FString >& OutRestrictedFolders
) const

   Misc/AutomationTest.h  

FString GetTagsForAutomationTest ( const FString& InTestName
)

 Fetch the tags associated with a test Misc/AutomationTest.h  

TArray< FString > GetTagsForAutomationTestAsArray ( const FString& InTestName
)

 Fetch the tags associated with a test as an array of strings Misc/AutomationTest.h  

void GetTestFullNamesMatchingTagPattern ( TArray< FString >& OutTestNames,
const FString& TagPattern
) const

 Collects all registered tests with associated tags matching the pattern Misc/AutomationTest.h  
Gets a scratch space location outside of the project and saved directories. Misc/AutomationTest.h  

void GetValidTestNames ( TArray< FAutomationTestInfo >& TestInfo
) const

 Populates the provided array with the names of all tests in the framework that are valid to run for the current application settings. Misc/AutomationTest.h  

bool IsAnyOnEnteringTestSectionBound()

   Misc/AutomationTest.h  

bool IsAnyOnLeavingTestSectionBound()

   Misc/AutomationTest.h  

bool IsLatentCommandQueueEmpty()

 Whether there is no latent command in queue Misc/AutomationTest.h  

bool IsTagImmutable ( const FString& InTestName,
const FString& InTag
) const

   Misc/AutomationTest.h  

void LoadTestModules()

 Load any modules that are not loaded by default and have test classes in them Misc/AutomationTest.h  

void LoadTestTagMappings ( const FString& BasePath,
const bool RestrictedPathsOnly
)

 Reads test tag mappings under a certain base path Misc/AutomationTest.h  
Reads test tag mappings from all Tags.ini config files Misc/AutomationTest.h  

void NotifyPerformanceDataRetrieved ( bool bSuccess,
const FString& ErrorMessage
)

   Misc/AutomationTest.h  

void NotifyScreenshotComparisonComplete ( const FAutomationScreenshotCompareResults& CompareResults
)

 Notify that the screenshot comparison has completed Misc/AutomationTest.h  

void NotifyScreenshotComparisonReport ( const FAutomationScreenshotCompareResults& CompareResults
)

 Notify the screenshot comparison report to the framework Misc/AutomationTest.h  

void NotifyScreenshotTakenAndCompared()

   Misc/AutomationTest.h  

void NotifyTestDataRetrieved ( bool bWasNew,
const FString& JsonData
)

   Misc/AutomationTest.h  

FOnTestScreenshotAndTraceCaptured & OnScreenshotAndTraceCaptured()

 Accessor for delegate called when a png screenshot is captured and a frame trace Misc/AutomationTest.h  

FOnTestScreenshotCaptured & OnScreenshotCaptured()

 Accessor for delegate called when a png screenshot is captured Misc/AutomationTest.h  

bool RegisterAutomationTest ( const FString& InTestNameToRegister,
FAutomationTestBase* InTestToRegister
)

 Register a automation test into the framework. Misc/AutomationTest.h  

bool RegisterAutomationTestTags ( const FString& InTestNameToRegister,
const FString& InTestTagsToRegister,
bool InImmutable
)

 Register tag metadata for a test into the framework. Misc/AutomationTest.h  

bool RegisterComplexAutomationTestTags ( const FAutomationTestBase* InTest,
const FString& InBeautifiedTestName,
const FString& InTestTagsToRegister
)

 Helper to register tags for a individual test enumerated by a Complex test Misc/AutomationTest.h  

void ResetTests()

 Reset status of worker (delete local files, etc) Misc/AutomationTest.h  

bool RunSmokeTests()

 Attempt to run all fast smoke tests that are valid for the current application configuration. Misc/AutomationTest.h  

void SaveTestTagMappings ( const TArray< FString >& TestFullNames,
const TArray< FString >& TestFilePaths,
const FBeforeTagMappingConfigSaved& BeforeConfigSaved,
const FAfterTagMappingConfigSaved& AfterConfigSaved
) const

 Saves test tag mappings optionally invoking before / after events. Misc/AutomationTest.h  

void SetCaptureStack ( bool bCapture
)

 Used to disabled stack capture when an error or warning event is triggered. Misc/AutomationTest.h  

void SetDeveloperDirectoryIncluded ( const bool bInDeveloperDirectoryIncluded
)

 Sets whether we want to include content in developer directories in automation testing Misc/AutomationTest.h  

void SetForceSmokeTests ( const bool bInForceSmokeTests
)

 Sets forcing smoke tests. Misc/AutomationTest.h  

void SetRequestedTestFilter ( const EAutomationTestFlags InRequestedTestFlags
)

 Sets which set of tests to pull from. Misc/AutomationTest.h  

bool ShouldTestContent ( const FString& Path
) const

 Whether the testing framework should allow content to be tested or not. Misc/AutomationTest.h  

void StartTestByName ( const FString& InTestToRun,
const int32 InRoleIndex,
const FString& InFullTestPath
)

 Attempt to start the specified test. Misc/AutomationTest.h  

bool StopTest ( FAutomationTestExecutionInfo& OutExecutionInfo
)

 Stop the current test and return the results of execution Misc/AutomationTest.h  

void TriggerOnEnteringTestSection ( const FString& Section
) const

   Misc/AutomationTest.h  

void TriggerOnLeavingTestSection ( const FString& Section
) const

   Misc/AutomationTest.h  

bool UnregisterAutomationTest ( const FString& InTestNameToUnregister
)

 Unregister a automation test with the provided name from the framework. Misc/AutomationTest.h  

bool UnregisterAutomationTestTags ( const FString& InTestNameToUnregister
)

 Unregister tags for a automation test from the framework. Misc/AutomationTest.h  

Static

Name Remarks Include Path Unreal Specifiers

static FAutomationTestFramework & Get()

 Return the singleton instance of the framework. Misc/AutomationTest.h  

static FAutomationTestFramework & GetInstance()

   Misc/AutomationTest.h  

static bool NeedLogBPTestMetadata()

 Whether to output blueprint functional test metadata to the log when test is running Misc/AutomationTest.h  

static bool NeedPerformStereoTestVariants()

 Whether to also run stereo test variants for screenshot functional tests Misc/AutomationTest.h  

static bool NeedSkipStackWalk()

 Whether to skip stack walk while iterating for listing the tests Misc/AutomationTest.h  

static bool NeedUseLightweightStereoTestVariants()

 Whether to skip variants when the baseline test fails, and skip saving screenshots for successful variants Misc/AutomationTest.h  

Operators

Name Remarks Include Path Unreal Specifiers
  Misc/AutomationTest.h  

Ask questions and help your peers Developer Forums

Write your own tutorials or read those from others Learning Library

On this page