Skip to main content

What is the benefit of writing to a temp location, And then copying it to the intended destination? [Resolved]

I am writing an application that works with satellite images, and my boss asked me to look at some of the commercial applications, and see how they behave. I found a strange behavior and then as I was looking, I found it in other standard applications as well.

These programs first write to the temp folder, and then copy it to the intended destination.

Example: 7zip first extracts to the temp folder, and then copies the extracted data to the location that you had asked it to extract the data to.

I see several problems with this approach:

  1. The temp folder might not have enough space, while the intended location might have that much space.

  2. If it is a large file, it can take a non-negligible amount of time for the copy operation.

I thought about it a lot, but I couldn't see one single positive point to doing this. Am I missing something, or is there a real benefit to doing this?

Asked January 11, 2017
Posted Under: Programming
3 Answers

A few reasons I can think of:

  • On most platforms, file moves are atomic, but file writes are not (especially if you can't write all the data in one go). So if you have the typical producer/consumer pattern (one process produces files, the other watches a directory and picks up everything it finds), writing to a temp folder first and only then moving to the real location means the consumer can never see an unfinished file.
  • If the process that writes the file dies halfway through, you have a broken file on your disk. If it's in a real location, you have to take care of cleaning it up yourself, but if it's in a temp location, the OS will take care of it.
  • If the file happens to be created while a backup job is running, the job may pick up an incomplete file; temp directories are generally excluded from backups, so the file will only be included once moved to the final destination.
  • The temp directory may be on a fast-but-volatile filesystem (e.g. a ramdisk), which can be beneficial for things like downloading several chunks of the same file in parallel, or doing in-place processing on the file with lots of seeks. Also, temp directories tend to cause more fragmentation than directories with less frequent reads, writes, and deletes, and keeping the temp directory on a separate partition can help keep fragmentation of the other partitions down.

TL;DR - it mostly boils down to atomicity, that is, you want to make it so that (at the final location) the file is either complete or not there at all at any given time.

Answered January 11, 2017
Dan Diplo's case is useful when updating an existing file. You don't want to replace the old one until the new one is successfully written, lest the write of the new one fail or something else read the new one before it's fully written. – RalphChapin Jun 18 '12 at 14:29
 CanDoerz  1 week ago
Sometime writing to a temp file is also useful when several instances of your program could try to upload the same file at the same time, and you cannot rely on the file system to provide the right locking level (like poorly-defined cloud storage API). Using temp file will make sure the resulting file won't end up with a mix of data from both uploads. Of course this is just another instance of atomicity. – Krzysztof Kozielczyk Jun 16 '12 at 19:18
 CanDoerz  1 week ago
@DanDiplo: On Windows, it probably doesn't matter either way... – tdammers Jun 16 '12 at 17:02
 CanDoerz  1 week ago
Some programs will extract/copy to the intended directory, but they will use a temporary file extension (e.g. .tmp) and rename it when complete. – Dan Diplo Jun 16 '12 at 16:43
 CanDoerz  1 week ago
If the temp directory is on a separate partition, you lose the atomicity. – yfeldblum Jun 16 '12 at 16:43
 CanDoerz  1 week ago

This seems to be an issue in Windows, more specifically related to how the drag-drop is managed.

The developers of the WINSCP client have developed their own shell extension, which overrides this drag-drop behavior and allows dropping the file to the right folder immediately. They explain the trick in their documentation and, more interesting, what is the problem and how they solved it.

Here is the interesting part:

Windows drag&drop mechanics does not allow source application of drag&drop operation to find out easily, where the files are dropped. It is up to target application (Windows Explorer usually) to transfer files to destination. It is rather reasonable, because source application can hardly transfer files to all possible destinations. Keep in mind that you can drop files not only to a directory, but even to ZIP file (or any other archive), remote directory (via FTP, SFTP, SCP, …), trash, …

Obviously even Windows Explorer (or any other target application, like WinZip) cannot download files from any possible source (particularly it does not know SFTP/SCP).

Also, specifically for 7Zip: user ray023 answers this question in the SuperUser Stack Q&A:

Basically, if instead of drag-dropping your file you use the "extract here" method available in bith 7-ZIP and Winrar, the files are directly extractly to the right directory.

Answered January 11, 2017
Good comment basically boils down to: why does a program use a temp folder? Because when drag and dropping it doesn't know the destination folder and just "gives it" to windows, then windows takes over and puts the file in the right place. – Pieter B Jun 29 '12 at 8:06
 CanDoerz  1 week ago
@DevdattaTengshe thank you – Jalayn Jun 17 '12 at 12:33
 CanDoerz  1 week ago
While your answer wasn't exactly what i was looking for, +1, for solving my problem with 7zip – Devdatta Tengshe Jun 17 '12 at 11:42
 CanDoerz  1 week ago
@Phil The example given is, from what I understand, what happens when you drag one file from an archive into a folder on the system, without explicitely asking to "extract it". Yet, I re-read the question, and yes, I think you are right in that the author wanted to know more about the utility of temporary folders, which by the way was masterfuly answered by tdammers (+1 voted by me too) – Jalayn Jun 16 '12 at 23:11
 CanDoerz  1 week ago
The question was not "Why does 7zip use temp files?" The actual question was "Why are temp files commonly used in software development?" – Phil Jun 16 '12 at 20:22
 CanDoerz  1 week ago

If you have to do any kind of data processing to the file (decode/convert/etc..), then it's better to use a temporary file and when completed, and only if completed, transfer the result to the final destination.


  1. Only completed files reach destination
  2. Temporary file may (should) reside in fast media
  3. Avoid fragmentation on final file
  4. Allows the use of other media as final destination (ftp, cloud, whatever)
  5. Aborted temp files are easier to clean

I don't see real benefits of writing straight to the destination while processing the data.

Answered January 11, 2017
Your Answer