安全的原子文件复制操作 [英] A safe, atomic file-copy operation

问题描述

我需要将文件从一个位置复制到另一个位置，并且如果文件已经存在于目标位置（不覆盖），则需要引发异常（或至少以某种方式可以识别）。

I need to copy a file from one location to another, and I need to throw an exception (or at least somehow recognise) if the file already exists at the destination (no overwriting).

我可以先使用os.path.exists（）进行检查，但是非常重要的一点是，不能在检查和复制之间的短时间内创建文件。

I can check first with os.path.exists() but it's extremely important that the file cannot be created in the small amount of time between checking and copying.

是否有内置的方法可以执行此操作，或者是否可以将动作定义为原子操作？

Is there a built-in way of doing this, or is there a way to define an action as atomic?

推荐答案

实际上是一种自动且安全地执行此操作的方法，前提是所有参与者都以相同的方式进行操作。它是无锁重击的改编-a-mole算法，而且不完全是琐碎的事情，因此，可以将 no作为一般答案；）

There is in fact a way to do this, atomically and safely, provided all actors do it the same way. It's an adaptation of the lock-free whack-a-mole algorithm, and not entirely trivial, so feel free to go with "no" as the general answer ;)

1. 检查文件是否已存在。


2. 生成唯一ID


3. 使用临时名称（例如< target>。< UUID> .tmp）将源文件复制到目标文件夹。 。


4. 重命名^†副本< target>-< UUID> .mole.tmp 。


5. 查找任何其他与模式匹配的文件 < target>-*。mole.tmp 。
   
   
   
   
   • 如果他们的UUID比您的UUID大，请尝试删除它。 （不必担心它是否消失了。）
   
   
   • 如果他们的UUID比您的UUID小，请尝试删除您自己的UUID。 （再次，不用担心它是否会消失。）从现在开始，将其UUID视为您自己的UUID。
   
   

1. Check whether the file already exists. Stop if it does.
2. Generate a unique ID
3. Copy the source file to the target folder with a temporary name, say, <target>.<UUID>.tmp.
4. Rename^† the copy <target>-<UUID>.mole.tmp.
5. Look for any other files matching the pattern <target>-*.mole.tmp.
   • If their UUID compares greater than yours, attempt to delete it. (Don't worry if it's gone.)
   • If their UUID compares less than yours, attempt to delete your own. (Again, don't worry if it's gone.) From now on, treat their UUID as if it were your own.

您完成了！

想象每个候选源文件都是从洞中冒出来的。中途退出时，它会暂停并重击所有竞争的痣，然后再检查没有其他痣完全出现。如果把它贯穿在脑海中，应该会看到只有一颗痣会一直排出。为了防止该系统动态锁定，我们添加了一个总顺序，以决定哪个痣可以重击。 am！ 博士学位论文 无锁算法。

Imagine each candidate source file is a mole coming out of its hole. Half-way out, it pauses and whacks any competing moles back into the ground, before checking no other mole has fully emerged. If you run this through in your head, you should see that only one mole will ever make it all the way out. To prevent this system from livelocking, we add a total ordering on which mole can whack which. Bam! A  PhD thesis  lock-free algorithm.

^†步骤4看起来似乎不必要-为什么不首先使用该名称？但是，另一个过程可能会在步骤5中采用您的  mole  文件，并在步骤7中使其成为赢家。因此，请务必不要写出内容，这非常重要！在同一文件系统上的重命名是原子的，因此第4步是安全的。

^† Step 4 may look unnecessary—why not just use that name in the first place? However, another process may "adopt" your  mole  file in step 5, and make it the winner in step 7, so it's very important that you're not still writing out the contents! Renames on the same file system are atomic, so step 4 is safe.

这篇关于安全的原子文件复制操作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！