为什么派生我的进程导致文件被无限读取 [英] Why does forking my process cause the file to be read infinitely
问题描述
我将整个程序简化为一个简短的主程序，该程序可以复制该问题，因此请原谅我没有任何意义.
I've boiled down my entire program to a short main that replicates the issue, so forgive me for it not making any sense.
input.txt是一个文本文件，其中包含几行文本.这个煮好的程序应该打印那些行.但是，如果调用fork，程序将进入一个无限循环，在该循环中，它将一遍又一遍地打印文件的内容.
input.txt is a text file that has a couple lines of text in it. This boiled down program should print those lines. However, if fork is called, the program enters an infinite loop where it prints the contents of the file over and over again.
据我了解，在本代码段中使用fork的方式本质上是禁止操作的.它分叉，父母等待孩子继续前进，然后孩子立即被杀死.
As far as I understand fork, the way I use it in this snippet is essentially a no-op. It forks, the parent waits for the child before continuing, and the child is immediately killed.
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(){
freopen("input.txt", "r", stdin);
char s[MAX];
int i = 0;
char* ret = fgets(s, MAX, stdin);
while (ret != NULL) {
//Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0) {
exit(0);
} else {
waitpid(pid, &status, 0);
}
//End region
printf("%s", s);
ret = fgets(s, MAX, stdin);
}
}
进一步的调查仅使我的问题变得陌生.如果文件包含< 4空行或< 3行文本，则文件不会中断.但是，如果不止于此，它会无限循环.
Further investigation has only made my issue stranger. If the file contains <4 blank lines or <3 lines of text, it does not break. However, if there are more than that, it loops infinitely.
Edit2:如果文件包含数字3行数字，它将无限循环，但是如果包含3行单词，则不会循环.
If the file contains numbers 3 lines of numbers it will infinitely loop, but if it contains 3 lines of words it will not.
推荐答案
我对出现问题感到惊讶，但在Linux上似乎确实是一个问题(我在运行于VMWare Fusion VM上的Ubuntu 16.04 LTS上进行了测试我的Mac)-但是在运行macOS 10.13.4(High Sierra)的Mac上这不是问题，我也不希望它在Unix的其他变体上也成为问题.
I am surprised that there is a problem, but it does seem to be a problem on Linux (I tested on Ubuntu 16.04 LTS running in a VMWare Fusion VM on my Mac) — but it was not a problem on my Mac running macOS 10.13.4 (High Sierra), and I wouldn't expect it to be a problem on other variants of Unix either.
正如我在
每个流后面都有一个打开的文件描述和一个打开的文件描述符.进程分叉时，子级拥有自己的一组打开文件描述符(和文件流)，但是子级中的每个文件描述符都与父级共享打开文件描述. IF (这是一个很大的'if')关闭文件描述符的子进程首先执行与 There's an open file description and an open file descriptor behind each stream. When the process forks, the child has its own set of open file descriptors (and file streams), but each file descriptor in the child shares the open file description with the parent. IF (and that's a big 'if') the child process closing the file descriptors first did the equivalent of 请参阅POSIX See POSIX 打开的文件描述符是进程专用的；打开的文件描述由初始打开文件"操作创建的文件描述符的所有副本共享.打开文件描述的关键属性之一是当前查找位置.这意味着子进程可以更改父进程的当前查找位置，因为它位于共享的打开文件描述中. The open file descriptors are private to a process; the open file descriptions are shared by all copies of the file descriptor created by an initial 'open file' operation. One of the key properties of the open file description is the current seek position. That means that a child process can change the current seek position for a parent — because it is in the shared open file description. 我使用了以下代码-原始代码的适度版本，可以使用严格的编译选项进行干净地编译: I used the following code — a mildly adapted version of the original that compiles cleanly with rigorous compilation options: 其中一项修改将循环数(子级)限制为仅30个.
我使用了一个数据文件，其中包含4行20个随机字母加一个换行符(总共84个字节): One of the modifications limits the number of cycles (children) to just 30.
I used a data file with 4 lines of 20 random letters plus a newline (84 bytes total): 我在Ubuntu的 I ran the command under 共有31个文件，文件名称的格式为 There were 31 files with names of the form 碰巧的是，前4个孩子每个都表现出四种行为中的一种-并且每4个孩子中的每一个表现出相同的模式. It just so happened that the first 4 children each exhibited one of the four behaviours — and each further set of 4 children exhibited the same pattern. 这表明，四分之三的孩子确实在退出之前对标准输入执行了 This shows that three out of four of the children were indeed doing an 此版本的代码使用单独的文件流(和文件描述符)和 This version of the code, using a separate file stream (and file descriptor) and 这也表现出相同的行为，只是发生查找的文件描述符是 This also exhibits the same behaviour, except that the file descriptor on which the seek occurs is IMO，这是一个错误.您应该不会遇到这个问题.
这很可能是Linux(GNU C)库而不是内核中的错误.这是由子进程中的 IMO, this is a bug. You should not be able to run into this problem.
It is most likely a bug in the Linux (GNU C) library rather than the kernel. It is caused by the GLIBC 错误23151 -包含未关闭文件的分叉进程在退出前不起作用并可能导致父I/O无限循环. GLIBC Bug 23151 - A forked process with unclosed file does lseek before exit and can cause infinite loop in parent I/O. 该错误创建于2019-05-08美国/太平洋地区，并于2018-05-09被关闭为INVALID.给出的原因是: The bug was created 2019-05-08 US/Pacific, and was closed as INVALID by 2018-05-09. The reason given was: 请阅读
http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02 .html#tag_15_05_01 ，
尤其是本段: Please read
http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01,
especially this paragraph: 请注意，在 Note that after a
所提到的POSIX的完整部分(不注意是C标准未涵盖的部分)是:
The complete section of POSIX referred to (apart from verbiage noting that this is not covered by the C standard) is this: 可以通过文件描述符访问打开的文件描述，该文件描述符是使用 An open file description may be accessed through a file descriptor, which is created using functions such as 可以通过显式用户操作来创建或破坏句柄，而不会影响基础打开文件的描述.创建它们的一些方法包括 Handles can be created or destroyed by explicit user action, without affecting the underlying open file description. Some of the ways to create them include 在可能影响文件偏移量的操作中从未使用过的文件描述符(例如， A file descriptor that is never used in an operation that could affect the file offset (for example, 涉及任何一个句柄(活动句柄")的函数调用的结果在POSIX.1-2017的此卷中的其他地方定义，但是如果使用两个或多个句柄，并且其中任何一个是流，应用程序应确保按照以下说明协调其行为.如果不这样做，则结果是不确定的. The result of function calls involving any one handle (the "active handle") is defined elsewhere in this volume of POSIX.1-2017, but if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined. 当 A handle which is a stream is considered to be closed when either an (1) [sic]使用'non-full'可能是'non-null'的错字. 要使一个句柄成为活动句柄，应用程序应确保在上一次使用该句柄(当前活动句柄)和第一次使用第二个句柄(将来的活动句柄)之间执行以下操作.然后，第二个手柄成为活动手柄.应用程序影响第一个句柄上文件偏移量的所有活动都应暂停，直到它再次成为活动文件句柄为止. (如果流功能具有影响文件偏移量的基础功能，则应将其视为影响文件偏移量.) For a handle to become the active handle, the application shall ensure that the actions below are performed between the last use of the handle (the current active handle) and the first use of the second handle (the future active handle). The second handle then becomes the active handle. All activity by the application affecting the file offset on the first handle shall be suspended until it again becomes the active file handle. (If a stream function has as an underlying function one that affects the file offset, the stream function shall be considered to affect the file offset.) 句柄不必在同一过程中应用这些规则. The handles need not be in the same process for these rules to apply. 请注意，在 Note that after a 对于第一个手柄，适用以下第一个适用条件.在采取以下必要的操作之后，如果句柄仍处于打开状态，则应用程序可以将其关闭. For the first handle, the first applicable condition below applies. After the actions required below are taken, if the handle is still open, the application can close it. 如果它是文件描述符，则无需执行任何操作. If it is a file descriptor, no action is required. 如果要对此打开文件描述符的任何句柄执行的唯一进一步操作是关闭它，则无需采取任何操作. If the only further action to be performed on any handle to this open file descriptor is to close it, no action need be taken. 如果它是无缓冲的流，则无需采取任何措施. If it is a stream which is unbuffered, no action need be taken. 如果它是行缓冲的流，并且写入该流的最后一个字节是 If it is a stream which is line buffered, and the last byte written to the stream was a 是该流上的最新操作)，无需采取任何操作. was the most recent operation on that stream), no action need be taken. 如果它是可以写入或追加(但也不能读取)的流，则应用程序应执行 If it is a stream which is open for writing or appending (but not also open for reading), the application shall either perform an 如果流已打开以供读取并且位于文件的末尾( If the stream is open for reading and it is at the end of the file ( 如果使用允许读取的模式打开流，并且基础打开文件描述引用的是能够搜索的设备，则应用程序应执行 If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an 第二个句柄: 如果在满足上面第一个句柄的要求之前停止活动句柄的访问，则打开文件描述的状态将变得不确定.在 If the active handle ceases to be accessible before the requirements on the first handle, above, have been met, the state of the open file description becomes undefined. This might occur during functions such as a The 遵循这些规则时，无论使用何种句柄顺序，实现都应确保应用程序，即使是由多个进程组成的应用程序，也应产生正确的结果:写入时不会丢失或重复任何数据，并且所有数据均应按顺序书写，除非寻求者要求.由实现定义，是否在所有条件下以及在什么条件下仅能一次看到所有输入. When these rules are followed, regardless of the sequence of handles used, implementations shall ensure that an application, even one consisting of several processes, shall yield correct results: no data shall be lost or duplicated when writing, and all data shall be written in order, except as requested by seeks. It is implementation-defined whether, and under what conditions, all input is seen exactly once. 每个在流上运行的函数都具有零个或多个底层函数".这意味着流功能与基础功能具有某些特征，但不要求流功能的实现与其基础功能之间存在任何关系. Each function that operates on a stream is said to have zero or more "underlying functions". This means that the stream function shares certain traits with the underlying functions, but does not require that there be any relation between the implementations of the stream function and its underlying functions.
很难读！如果您不清楚打开文件描述符和打开文件描述之间的区别，请阅读
That is hard reading! If you're not clear on the distinction between open file descriptor and open file description, read the specification of 在此问题代码的上下文中(以及在读取文件时创建不需要的子进程)，我们打开一个文件流句柄以仅读取尚未遇到EOF的文件(因此 In the context of the code in this question (and also for Unwanted child processes being created while file reading), we have a file stream handle open for reading only which has not yet encountered EOF (so 规范的关键部分之一是:应用程序应完全为 One of the crucial parts of the specification is: The application shall prepare for a 这意味着为第一个文件句柄"概述的步骤是相关的，并逐步执行，第一个适用条件是最后一个: This means that the steps outlined for 'first file handle' are relevant, and stepping through them, the first applicable condition is the last: 如果您查看 If you look at the definition for 如果 stream 指向未输入最新操作的输出流或更新流，则 If stream points to an output stream or an update stream in which the most recent operation was not input, 对于一个打开的带有基础文件描述的流，如果该文件尚未在EOF上，并且该文件具有查找能力，则应将基础打开文件描述的文件偏移量设置为的位置.流，并通过 For a stream open for reading with an underlying file description, if the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream, and any characters pushed back onto the stream by 尚不清楚如果将 It isn't exactly clear what happens if you apply The 因此，对于该问题中的代码，看来 So, for the code in this question, it seems that 这篇关于为什么派生我的进程导致文件被无限读取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！lseek(fd, 0, SEEK_SET)等效的操作，然后还将定位该文件描述符的子描述符.父进程，这可能导致无限循环.但是，我从来没有听说过可以这样做的图书馆.没有理由这样做.
lseek(fd, 0, SEEK_SET), then that would also position the file descriptor for the parent process, and that could lead to an infinite loop. However, I've never heard of a library that does that seek; there's no reason to do it.open() 和open() and fork() for more information about open file descriptors and open file descriptions.#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(void)
{
if (freopen("input.txt", "r", stdin) == 0)
return 1;
char s[MAX];
for (int i = 0; i < 30 && fgets(s, MAX, stdin) != NULL; i++)
{
// Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0)
{
exit(0);
}
else
{
waitpid(pid, &status, 0);
}
// End region
printf("%s", s);
}
return 0;
}
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
strace下运行了命令:strace on Ubuntu:$ strace -ff -o st-out -- neof97
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
…
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
$
st-out.808##，其中的哈希值为2位数字.主过程文件很大.其他的则很小，尺寸为66、110、111或137:st-out.808## where the hashes were 2-digit numbers. The main process file was quite large; the others were small, with one of the sizes 66, 110, 111, or 137:$ cat st-out.80833
lseek(0, -63, SEEK_CUR) = 21
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80834
lseek(0, -42, SEEK_CUR) = -1 EINVAL (Invalid argument)
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80835
lseek(0, -21, SEEK_CUR) = 0
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80836
exit_group(0) = ?
+++ exited with 0 +++
$
lseek().显然，我现在已经看到了一个图书馆.我不知道为什么它被认为是一个好主意，但是从经验上讲，这就是正在发生的事情.lseek() on standard input before exiting. Obviously, I have now seen a library do it. I have no idea why it is thought to be a good idea, though, but empirically, that is what is happening.fopen()而不是freopen()也会出现问题.fopen() instead of freopen() also runs into the problem.#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(void)
{
FILE *fp = fopen("input.txt", "r");
if (fp == 0)
return 1;
char s[MAX];
for (int i = 0; i < 30 && fgets(s, MAX, fp) != NULL; i++)
{
// Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0)
{
exit(0);
}
else
{
waitpid(pid, &status, 0);
}
// End region
printf("%s", s);
}
return 0;
}
3而不是0.因此，我的两个假设被驳斥了-与freopen()和stdin有关；两者在第二个测试代码中均显示为错误.3 instead of 0. So, two of my hypotheses are disproven — it's related to freopen() and stdin; both are shown incorrect by the second test code.lseek()引起的.尚不清楚(因为我没有去看源代码)该库在做什么或为什么这样做.lseek() in the child processes. It is not clear (because I've not gone to look at the source code) what the library is doing or why.
fork()之后，存在两个句柄，其中一个以前存在. […] fork(), two handles exist where one existed before. […] POSIX
POSIX
2.5.1文件描述符和标准I/O流的交互
pipe() ，或通过使用 fopen() 之类的函数创建的流或 popen() .在它所指向的打开文件描述中，文件描述符或流都称为句柄".打开的文件描述可能具有多个句柄.
2.5.1 Interaction of File Descriptors and Standard I/O Streams
open() or pipe(), or through a stream, which is created using functions such as fopen() or popen(). Either a file descriptor or a stream is called a "handle" on the open file description to which it refers; an open file description may have several handles.fcntl() ， dup() ， fileno() 和 fork() .它们至少可以被 fclose() ， close() 和fcntl(), dup(), fdopen(), fileno(), and fork(). They can be destroyed by at least fclose(), close(), and the exec functions.read() ， write() ，或 lseek() )不被视为该讨论的句柄，但可能会产生一个(例如，由于 fdopen() ， dup() 或 fopen() 或 fdopen() ，只要不使用它直接由应用程序影响文件的偏移量. read() 和 lseek() 明确地影响了它.read(), write(), or lseek()) is not considered a handle for this discussion, but could give rise to one (for example, as a consequence of fdopen(), dup(), or fork()). This exception does not include the file descriptor underlying a stream, whether created with fopen() or fdopen(), so long as it is not used directly by the application to affect the file offset. The read() and write() functions implicitly affect the file offset; lseek() explicitly affects it.freopen() 且不完整<在其上执行sup>(1)文件名(对于 close() ，fclose(), or freopen() with non-full(1) filename, is executed on it (for freopen() with a null filename, it is implementation-defined whether a new handle is created or the existing one reused), or when the process owning that stream terminates with exit(), abort(), or due to a signal. A file descriptor is closed by close(), _exit(), or the exec() functions when FD_CLOEXEC is set on that file descriptor.
fork() 之后，存在两个句柄，其中一个以前存在.如果两个句柄都可以访问，则应用程序应确保它们都处于另一个可以首先成为活动句柄的状态.应用程序应完全为 fork() 作准备.更改活动手柄. (如果其中一个进程执行的唯一操作是 exec() 函数或 _exit() (不是fork(), two handles exist where one existed before. The application shall ensure that, if both handles can ever be accessed, they are both in a state where the other could become the active handle first. The application shall prepare for a fork() exactly as if it were a change of active handle. (If the only action performed by one of the processes is one of the exec() functions or _exit() (not exit()), the handle is never accessed in that process.)
<newline>(即，就像a:<newline> (that is, as if a:putc('\n')
fflush(), or the stream shall be closed.feof() 为true)，则无需采取任何操作.feof() is true), no action need be taken.fflush(), or the stream shall be closed.
fseek() (根据句柄的类型)到适当的位置.fork() 或 _exit() .fork() or _exit().exec() 函数使所有打开的流无法访问在调用它们时，与新的过程映像可能使用哪些流或文件描述符无关.exec() functions make inaccessible all streams that are open at the time they are called, independent of which streams or file descriptors may be available to the new process image.训ege
open()和fork()(以及dup()或文件描述符和Exegesis
open() and fork() (and dup() or dup2()). The definitions for file descriptor and open file description are also relevant, if terse.feof()不会返回true，即使读取位置位于文件的末尾).feof() would not return true, even though the read position is at the end of the file).fork()作准备，就像它是活动句柄的更改一样. fork() exactly as if it were a change of active handle.
fflush()，否则应关闭流.
fflush(), or the stream shall be closed.fflush() 的定义，找到:fflush(), you find:
fflush()将导致该流的任何未写入数据写入文件，[ CX]⌦，并标记基础文件的最后数据修改和最后文件状态更改时间戳.
fflush() shall cause any unwritten data for that stream to be written to the file, [CX] ⌦ and the last data modification and last file status change timestamps of the underlying file shall be marked for update.ungetc() 将所有字符推回到流中或 ungetwc() 后来未从流中读取的丢弃(无需进一步更改文件偏移量). ⌫ungetc() or ungetwc() that have not subsequently been read from the stream shall be discarded (without further changing the file offset). ⌫fflush()应用于与不可搜索文件关联的输入流会发生什么，但这不是我们的直接关注.但是，如果要编写通用库代码，则可能需要在流上执行fflush()之前，先了解基础文件描述符是否可搜索.或者，使用fflush(NULL)让系统执行所有I/O流所需的所有操作，请注意，这将丢失任何后推字符(通过ungetc()等).fflush() to an input stream associated with a non-seekable file, but that isn't our immediate concern. However, if you're writing generic library code, then you might need to know whether the underlying file descriptor is seekable before doing a fflush() on the stream. Alternatively, use fflush(NULL) to have the system do whatever is necessary for all I/O streams, noting that this will lose any pushed-back characters (via ungetc() etc).strace输出中显示的lseek()操作似乎正在实现fflush()语义，它将打开的文件描述的文件偏移量与流的文件位置相关联.lseek() operations shown in the strace output seem to be implementing the fflush() semantics associating the file offset of the open file description with the file position of the stream.fflush(stdin)在fork()之前是必需的，以确保一致性.不这样做会导致不确定的行为(如果不这样做，则结果是不确定的")，例如无限循环.fflush(stdin) is necessary before the fork() to ensure consistency. Not doing that leads to undefined behaviour ('if this is not done, the result is undefined') — such as looping indefinitely.
