Controlling Temporary File Storage for Windows Clients

During cloning, Windows clients write bulk loader data to multiple temporary text files for each data table being loaded. These temporary text files are used as holding areas for the bulk loader data. The Windows client uses overlapped operations to write data to one set of text files while the bulk loader is loading another set of files. The configuration file parameter max_temp_storage determines the maximum amount of storage to be used by all of the temporary files.

The DATABridge Client writes data to as many temporary files as it needs, while keeping track of the amount of storage used. When the amount of used storage exceeds half of the configured value of the configuration file parameter, max_temp_storage (default value is 40 MB), the DATABridge Client closes all the temporary files and queues them on the bulk loader thread’s work queue. While the bulk loader thread is sequentially launching the loads for these files (which run as separate processes), the DATABridge Client starts filling a new set of temporary files for the next group of loads. This mode of operation significantly enhances performance on systems that have more than one CPU.

You may want to adjust the value of this parameter depending on how fast the files get filled. Higher values reduce the amount of parallelism, and therefore, have a negative impact on performance. Conversely, smaller values may negatively impact performance by firing too many bulk loader operations.