1. Epic GamesDeveloper
  3. Derived Data Cache

Derived Data Cache

Overview of the Derived Data Cache and the storage of assets in formats used by Unreal and its target platforms

Derived Data Cache

DDC stands for Derived Data Cache.

Many Unreal Engine Assets require additional "derived data" before they can be used. (A simple example would be a Material that has a shader). Before the Material can be rendered, the shader must be compiled for the platform the editor is running on.

Because the derived data is large, and at times may need regenerating, it is not checked into source control. Instead it's kept in the Derived Data Cache (DDC).

Where the DDC Is Stored

Depending on how your project and system are configured, there can be several DDC caches in a hierarchy that ranges from fast to slow. When assessing derived data, your system will do the following to determine how quickly it can access it:

  1. When a piece of derived data is needed, the fastest cache is checked first, then the next fastest, and so on, until the data is found.
  2. When the data is found, it is copied into the fastest local cache so that it is quicker to access next time.
  3. If the data is not found, then it is first generated, then asynchronously copied into the caches so it is available for you (and potentially your team) in the future.

Content stored in the DDC is disposable, and can be regenerated at any time using the data stored in the .uasset file. Storing these derived formats externally makes it possible to easily add or change the formats used by the engine without needing to modify the source Asset file.

Types of DDCs

Typically, you will have at least the following two caches:

  1. Boot DDC: Startup files that are loaded into memory to increase boot times.
    1. For Example: ProjectDir/DerivedDataCache/Boot.ddc
  2. Local DDC: Derived data for your projects.
    1. For Example: EngineDir/DerivedDataCache

However, your project may also be set up to use a DDC Pak or Shared DDC.

DDC Pak

If you download Unreal Engine from the Epic Games Store, the engine will come with a DDC Pak (.ddp). The DDC Pak contains derived data for all engine content, so you can start working without compiling shaders and so on. Similarly, some samples are shipped with a DDC Pak for the same reason.

  1. Engine DDC Pak
    1. For Example: EngineDir/DerivedDataCache/Compressed.ddp
  2. Project DDC Pak
    1. For Example: ProjectDir/DerivedDataCache/Compressed.ddp

Shared DDC

One of our strongest recommendations for teams in the same location is that they set up a shared DDC. This is a network drive that all team members and build machines can read/write to. This then writes off the cost of creating any required DDC data across the entire team. For example, when an artist edits a shader, the DDC data will be written straight to the DDC share.

  • Shared DDC: Will be either a network or mapped drive.

An example of a shared drive would be: \epicgames.net\root\DDC-Global-Fortnite

Using a Shared DDC

A studios should use a shared DDC that all users in a particular location can access. This way, only one person needs to build the derived Asset format(s) and they will be automatically available to all other users. There will occasionally be stalls when Assets need to be processed, but the results are stored and shared. Even on a fairly small team, sharing Asset processing work in this way will eliminate most processing time.

It is not advised to copy an entire DDC across the internet, or back up your DDC, or restore a DDC from a backup. While not necessarily harmful, this would be a waste of time—it takes longer to transfer the amount of data stored in the DDC than it does to generate it from scratch locally. If you have a large project and want to distribute pre-built DDC data, you should generate a DDC Pak.

Setting Up Shared DDC

As you may see in BaseEngine.ini, the editor is already set up to use a Shared DDC that can be enabled one of three ways.

Here are the options from most to least recommended.

  1. Add an override (as pictured below) in the DefaultEngine.ini for your project that sets the path to a valid location for your team.
	[DerivedDataBackendGraph] Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=\\YourCompanyServer\DDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)

  1. Set an environment variable (as pictured below) of UE-SharedDataCachePath (On Mac/Linux UE_SharedDataCachePath) to the folder to use.

  2. In the editor, set the SharedDerivedDataCache variable.

Click image for full size.

You don't have to do anything special to \YourCompanyServer\DDC; as long as it is writable, the engine will create the structure it needs.

Disabling a Shared DDC

Developers working remotely with a project that has a shared DDC configured may experience poor performance due to the fact that it takes longer to access DDC data than it does to generate it. To temporarily disable a shared DCC, use one of the methods below:

  • Pass -ddc=noshared on the command line.
  • Set the environment variable to a local drive hard drive:
    • UE-SharedDataCachePath=None
    • On Mac: UE_SharedDataCachePath=None

Building Derived Data

The user who imports the Asset is the one who builds the derived data since they will most likely be using and testing that Asset in the Engine. However, there may be times when a new Asset needs to be processed. This happens automatically on an as-needed basis and should not result in much of an impact when running on fast hardware, though there may be occasional stalls.

You can fill your DDCs at any time by running the following command:

	UE4\Engine\Binaries\Win64\UE4Editor.exe ProjectName -run=DerivedDataCache -fill

Epic Games does this on a nightly basis to ensure that the DDC is always primed, but it is not necessary as the general automatic caching feature should suffice.

Mounting Your DDC Folder

For the DDC to be used over your internal network, the folder that contains the DDC must be mounted to a network drive. If you are unfamiliar with how to do this, see Map A Network Drive on Windows.

Distributing with a DDC

Cooking is the preferred method for packaging games since cooked builds do not need or use a DDC. However, DDCs can be packaged for distribution if the need arises.

To package a DDC:

  1. Run UE4Editor.exe from the UE4/Engine/Binaries/Win64 directory, passing the arguments shown below:
	UE4Editor.exe ProjectName -run=DerivedDataCache -fill -DDC=CreatePak
  1. This creates a DDC.ddp file in the UE4\ProjectName\DerivedDataCache directory.
  2. The engine automatically detects and uses the .ddp file.

DDC Settings

Where are DDC settings kept?

DDC settings are kept in DefaultEngine.ini under the [DerivedDataBackendGraph] section. You can view the defaults in BaseEngine.ini.

	[DerivedDataBackendGraph]
	; Keep files for at least 7 days
	MinimumDaysToKeepFile=7
	; Configure the root entry. It uses a KeyLength node to hash long strings then passes requests to AsyncPut
	Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
	; Configure the AsyncPut entry. It uses an AsyncPut node that then passes requests to Hierarchy
	AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
	; Configure the hierarchy entry. This uses multiple nodes that are used in order until a read is found (writes go to all writable entries)
	Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Shared)
	; Configure the Boot node. This holds 512MB of data used to speed up booting.
	Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
	; Configure the local node. This is a filesystem node with the following set -
	;	Readonly: Can data be written to this layer
	;	Clean: Perform a cleanup of old files on launch
	;	Flush: Nuke the DDC and start over
	;	PurgeTransient: Don't keep transient data in the DDC
	;	DeleteUnused: Clean up old files (happens on a background thread)
	;	UnusedFileAge: Age after which files are removed
	;	FoldersToClean: Max number of folders to clean in a session. -1 = Unlimited
	;	MaxFileChecksPerSec: How many files to check a second.
	;	Path: path to use for the filesystem DDC
	;	EnvPathOverride: An environment variable that if set will be used instead of path. E.g. UE-LocalDataCachePath=d:\DDC. ('None' disables the DDC)
	;	CommandLineOverride: A command line argument used in preference to the default / envvar setting. E.g. -SharedDataCachePath=\\someshare\folder
	;	EditorOverrideSetting: Editor user setting that overrides the default/envvar/command line values
	Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=34, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath, EditorOverrideSetting=LocalDerivedDataCache)
	; Configure the shared DDC that is accessed after local. It's a filesystem DDC and the parameters are explained above
	Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache, CommandLineOverride=SharedDataCachePath)
	; Configure an alternate shared DDC that is accessed after local. It's a filesystem DDC and the parameters are explained above
	AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=23, FoldersToClean=10, MaxFileChecksPerSec=1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
	; Configure a Project Pak node. This is a pre-generated DDC data file for the project that can be distributed to reduce runtime fetches/generation
	; See documentation for how to create a DDP via the DerivedDataCache commandlet
	Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
	; Configure a Project Pak node. This is a pre-generated DDC data file for the engine that can be distributed to reduce runtime fetches/generation
	EnginePak=(Type=ReadPak, Filename=%ENGINEDIR%DerivedDataCache/DDC.ddp)

Common Questions

Q: Can I have multiple DDC settings?

Yes! Just create a new [YourDDCSettings] entry in DefaultEngine.ini and then run the editor with -ddc=YourDDCSettings.

At Epic, we do this for three reasons:

  1. For teams in offices that want to use their own DDC instead of the global one.
  2. When creating DDC Paks we use one of several options that control what goes into the .pak file, such as [CreateInstalledEnginePak].
  3. For people working from home who don't want to use a Shared DDC due to slower internet or not having VPN enabled.

You'll see that BaseEngine.ini contains several DDC entries by default, including NoShared, which is used for #3 above (-ddc=NoShared).

Q: I am short on disk space. Can I move the Local DDC elsewhere?

Yes! You can adjust this in your project under the Editor Preferences. Aditionaly you can set an environment variable of UE-LocalDataCachePath to a path of your choice. For example: UE-LocalDataCachePath=d:\DDC.

  1. At the command prompt, type "setx UE-LocalDataCachePath d:\DDC".
  2. Restart Unreal Engine, along with any application you launch it from, such as Epic Launcher, UGS, or Visual Studio.

This can be used when you are short of space on your primary drive, or when you are working on multiple branches and wish to ensure the data isn't duplicatednote

Q: I have a slow network connection. Can I turn off the Shared DDC?

Yes! There are two options, depending on whether you want this to be temporary or permanent:

  • If you set the environment variable for a DDC to None it will disable it. In this case, you would set UE-SharedDataCachePath=None.
  • Launch the editor with -ddc=noshared.

Q: Can I change the Shared DDC path?

Yes! You can change it by setting the UE-SharedDataCachePath mentioned above.

However, you should never set UE-SharedDataCachePath to a local path! Doing this means you would have both Local and Shared caches on your machine—twice the disk space for zero gain!

Q: How can I diagnose DDC issues?

If you think your editor is not correctly reading DDC data, then search your logfile for LogDerivedDataCache.

	LogDerivedDataCache: Display: Max Cache Size: 512 MB
	LogDerivedDataCache: Loaded boot cache 0.07s 118MB ../../../EngineTest/DerivedDataCache/Boot.ddc.
	LogDerivedDataCache: Display: Loaded Boot cache: ../../../EngineTest/DerivedDataCache/Boot.ddc
	LogDerivedDataCache: FDerivedDataBackendGraph: Pak pak cache file ../../../EngineTest/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
	LogDerivedDataCache: Unable to find inner node Pak for hierarchical cache Hierarchy.
	LogDerivedDataCache: FDerivedDataBackendGraph: EnginePak pak cache file ../../../Engine/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
	LogDerivedDataCache: Unable to find inner node EnginePak for hierarchical cache Hierarchy.
	LogDerivedDataCache: Found environment variable UE-LocalDataCachePath=d:\DDC
	LogDerivedDataCache: Using Local data cache path d:\DDC: Writable
	LogDerivedDataCache: Using Shared data cache path \\epicgames.net\root\DDC-Global-UE4: Writable

Here you can see:

  • There's no project or Engine Pak cache, which is what would be expected as I compiled this build myself from Perforce.
  • My local data cache path is mapped to d:\DDC and is writable.
  • I am using the Epic shared cache, which is writable.

You can also run with -logcmds="LogDerivedDataCache Verbose" to turn on verbose logging.

Q: What is the S3 DDC?

The Fortnite DDC is extremely large due to the amount of content we have. When Covid19 struck, we quickly realized that it was very inefficient for developers to pull all this data from a network share over the limited bandwidth of a VPN to the office, so instead we distribute the most commonly used data via S3.

Setting up the engine to use the S3DDC is extremely easy, but you will need to set up automation that generates the DDC content and uploads it to S3.

Q: What is IDDCUtilsModuleInterface?

This interface allows you to use ? Name arguments in your DDC settings that can then be resolved dynamically at runtime based on platform or domain.

Unfortunately, we don't ship an example of this, but implementing it is straightforward:

	#include "CoreMinimal.h"
	#include "DerivedDataUtilsInterface.h"
	#include "Modules/ModuleManager.h"

	class FDCCUtilsModule : public IDDCUtilsModuleInterface
	{
			/** IDDCUtilsModuleInterface implementation */
			virtual FString GetSharedCachePath(const FString& CacheName) override;

			/** IModuleInterface implementation */
			virtual void StartupModule() override;

			/** List of internal DDC servers */
			TMap<FString, FString> NameToServerMap;
	}

	IMPLEMENT_MODULE(FDDCUtilsModule, DDCUtils)

	void FDDCUtilsModule::StartupModule()
	{
	#if PLATFORM_MAC
		NameToServerMap.Add(TEXT("?EpicDDC"), TEXT("/Volumes/UE4DDC"));
		NameToServerMap.Add(TEXT("?EpicDDC2"), TEXT("/Volumes/UE4DDC2"));
		NameToServerMap.Add(TEXT("?EpicSeaDDC"), TEXT("/Volumes/DDC"));

	#elif PLATFORM_LINUX
		NameToServerMap.Add(TEXT("?EpicDDC"), TEXT("/mnt/UE4DDC"));
		NameToServerMap.Add(TEXT("?EpicDDC2"), TEXT("/mnt/UE4DDC2"));
		NameToServerMap.Add(TEXT("?EpicSeaDDC"), TEXT("/mnt/DDC"));

	#else
		NameToServerMap.Add(TEXT("?EpicDDC"), TEXT("\\\\epicgames.net\\root\\DDC-Global-UE4"));
		NameToServerMap.Add(TEXT("?EpicDDC2"), TEXT("\\\\epicgames.net\\root\\UE4DDC2"));
		NameToServerMap.Add(TEXT("?EpicSeaDDC"), TEXT("\\\\sea.epicgames.net\\root\\DDC"));
	#endif
	}

	FString FDDCUtilsModule::GetSharedCachePath(const FString& CacheName)
	{
		FString* ServerName = NameToServerMap.Find(CacheName);
		if (ServerName)
		{
			return *ServerName;
		}
		else
		{
			return FString();
		}
	}

Q: How do I create a DDC Pak?

Creating a project-specific DDC:

  • UE4Editor.exe ProjectName -run=DerivedDataCache -fill -DDC=CreatePak

This commandlet will create a DDC for all content in the project. Alternatively, you can simply supply -DDC=CreatePak while running automation or even a user session to generate a more targeted set of content.

Creating an Engine DDC:

  • UE4Editor.exe -run=DerivedDataCache -fill -DDC=CreatePak

Ask questions and help your peers Developer Forums

Write your own tutorials or read those from others Learning Library

On this page