如何同步(使原子)从两个进程写入一个文件? [英] How can I synchronize -- make atomic -- writes on one file from from two processes?

问题描述

我有两个进程，每个进程都写入大量数据，并且我想控制将这些进程的写入同步到一个文件中.

包括(A1，A2，A3)的处理1写入缓冲器A和包括(B1，B2，B3)的处理2写入缓冲器B.当我们使用write()系统调用将这些缓冲区写入磁盘到同一文件(一次整个缓冲区:write(fd, A, sizeof(A)))时，文件架构如何?

• 是这样的吗:A，B或B，也许是A?
• 或者可能是这样:A1，A2，B1，A3，...

我问这个是因为系统调用是原子的.如果我们正在写入的数据缓冲区太大，会发生什么情况.就像普通磁盘文件的管道一样吗?

解决方案

如果要同时显示两个缓冲区的内容，则必须打开设置了O_APPEND标志的文件.附加标志在写入之前将查找到文件的末尾.如果没有此设置，则两个进程都可能指向文件的相同或重叠区域，最后写入的人将覆盖另一个写入的内容.

对write的每次调用都会写出所请求的字节数.如果您的进程被信号中断，那么您可能会以部分写入而告终-返回实际写入的字节数.无论是否写入所有字节，都将写入文件的一个连续部分.您没有得到第二种可能性提到的交错效果(例如A1，B1，A2，B2，...).

如果仅得到部分写入，则如何进行取决于您.您可以继续写入(从缓冲区开始偏移先前写入的字节数)，也可以放弃其余的写入.只有这样，您才能潜在地获得交错效果.

如果在另一个进程写之前完成一个写的内容很重要，那么在尝试写入任何数据之前，您应该考虑锁定文件以进行独占写访问(两个进程都必须检查)./p>

I have two process each writing large buffer of data, and I want to control synchronize those processes' writes to one file.

process 1 writing buffer A including (A1, A2, A3) and process 2 writing buffer B including (B1, B2, B3). when we use write() system call to write these buffers to disk to the same file(whole buffer at one time: write(fd, A, sizeof(A))) , How is the file schema?

• Is it like this: A, B or B, A maybe?
• or it could be like this: A1, A2, B1, A3, ...

I'm asking this because system calls are atomic. what happens if the data buffer we are writing is too large. Is it like pipes for regular disk files?

解决方案

If you want the contents of both buffers to be present, you have to open the files with the O_APPEND flag set. The append flag seeks to the end of the file before writing. Without this set, it's possible that both processes will be pointing to the same or overlapping areas of the file and whoever writes last will overwrite what the other has written.

Each call to write will write up to the number of bytes requested. If your process is interrupted by a signal, then you can end up with a partial write -- the actual number of bytes written is returned. Whether you get all of your bytes written or not, you'll have written one contiguous section of the file. You don't get the interleaving effect you mentioned as your second possibility (e.g. A1,B1,A2,B2,...).

If you only get a partial write, how you proceed is up to you. You can either continue writing (offset from the buffer start by the number of bytes previously written), or you can abandon the rest of your write. Only in this way could you potentially get the interleaving effect.

If it's important to have the contents of one write complete before the other process writes, then you should look into locking the file for exclusive write access (which both processes will have to check for) before attempting to write any data.

这篇关于如何同步(使原子)从两个进程写入一个文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！