由于缓冲导致的死锁.它是如何工作的? [英] Deadlocks due to buffering. How does it work?

问题描述

以下问题是对此答案中有关死锁的评论的回应.我很好奇死锁是如何产生的，因此我创建了一个测试程序:有一个父进程将大量数据写入子级STDIN，而子进程也将大量数据写入父级的读取器句柄.事实证明，如果数据大小超过80K(Ubuntu 16.04)，将会出现死锁:

The following question is a response to a comment about deadlocking in this answer. I am curious how deadlocks come about, so I created a test program: There is a parent process that writes a lot of data to the childs STDIN, whereas the child also writes a lot of data to the parent's reader handle. It turns out that there will be a deadlock if the data size exceeds 80K (Ubuntu 16.04):

parent.pl :

use feature qw(say);
use strict;
use warnings;

use IPC::Open2;

my $test_size = 80_000; # How many bytes to write?
my $pid = open2( my $reader, my $writer, 'child.pl' );

my $long_string = '0123456789' x ($test_size / 10);
printf "Parent: writing long string ( length: %d )\n", length $long_string;
print $writer $long_string;
close $writer;
say "Parent: Trying to read childs ouput..";
my $output = do { local $/; <$reader> };
printf "Parent: Got output with length %d..\n", length $output;
close $reader;
say "Parent: Reaping child..";
waitpid( $pid, 0 );
my $child_exit_status = $? >> 8;
say "Parent: Child exited with status: $child_exit_status";

child.pl :

use feature qw(say);
use strict;
use warnings;

my $test_size = 80_000; # How many bytes to write?
my $child_log_filename = 'childlog.txt';

open ( my $log, '>', $child_log_filename ) or die "Could not create log file: $!";

say $log "Child is running..";

my $long_string = '0123456789' x ($test_size / 10);
say $log "Length of output string: " . length $long_string;
say $long_string;

my $input = do { local $/; <STDIN> };
say $log "Length of input string: " . length $input;
exit 2;

为什么该程序会死锁?

推荐答案

出现死锁的原因似乎是在写入时管道被填满，而没有人从中读取.根据 pipe(7):

The reason for the deadlock seems to be that the pipes get filled up when writing and no one reads from them. According to pipe(7):

如果某个进程尝试写入一个完整的管道(请参见下文)，则 write(2)阻塞，直到从管道读取了足够的数据为止 允许写操作完成.

If a process attempts to write to a full pipe (see below), then write(2) blocks until sufficient data has been read from the pipe to allow the write to complete.

管道的容量有限.如果管道已满，则执行write(2) 将阻止还是失败，具体取决于是否设置了O_NONBLOCK标志. 不同的实现对管道容量有不同的限制. 应用程序不应依赖于特定的能力:一个应用程序 应该设计成使读取过程尽快消耗数据 它可用，因此不会阻止写入过程.

A pipe has a limited capacity. If the pipe is full, then a write(2) will block or fail, depending on whether the O_NONBLOCK flag is set. Different implementations have different limits for the pipe capacity. Applications should not rely on a particular capacity: an application should be designed so that a reading process consumes data as soon as it is available, so that a writing process does not remain blocked.

在2.6.11之前的Linux版本中，管道的容量是相同的 作为系统页面大小(例如，在i386上为4096字节).自Linux 2.6.11，管道容量为65536字节.从Linux 2.6.35开始，默认管道容量为65536字节

In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11, the pipe capacity is 65536 bytes. Since Linux 2.6.35, the default pipe capacity is 65536 bytes

因此，父级将一个长字符串写入子级的STDIN管道，该管道被填满，从而导致父级阻塞.同时，孩子正在给父母的阅读器管道写一个长字符串，这也被填满了，孩子被阻塞了.因此，子级正在等待父级从其读取器管道读取，而父级正在等待子级从其STDIN管道读取.所谓的僵局.

Hence, the parent writes a long string to the child's STDIN pipe, the pipe gets filled up, which causes the parent to block. At the same time the child is writing a long string to parents reader pipe, this also gets filled up and the child blocks. So the child is waiting for the parent to read from its reader pipe, while the parent is waiting for the child to read from its STDIN pipe. A so-called deadlock.

我尝试通过使用select，fcntl，sysread和syswrite解决此问题.由于syswrite将在我们尝试写入的内容超出管道容量的情况下阻塞，因此我使用fcntl来使写入器处理非阻塞状态.在这种情况下，syswrite将尽可能多地写入管道，然后立即返回其实际写入的字节数.

I tried for fun to solve this by using select, fcntl, sysread and syswrite. Since syswrite will block if we try to write more than the capacity of the pipe, I used fcntl to make the writer handle non blocking. In that case, syswrite will write as much as possible to the pipe, and then return immediately with the number of bytes it actually wrote.

注意:仅需要更改parent.pl(这也是需要的，因为我们不应假定可以访问孩子的源).这是 修改后的parent.pl可以防止死锁:

Note: only parent.pl needs to be changed (which is also desired, since we should not assume to have access to the child's source). Here is the modified parent.pl that will prevent the deadlock:

use feature qw(say);
use strict;
use warnings;

use Errno qw( EAGAIN );
use Fcntl;
use IO::Select;
use IPC::Open2;

use constant READ_BUF_SIZE => 8192;
use constant WRITE_BUF_SIZE => 8192;

my $test_size = 80_000; # How many bytes to write?
my $pid = open2( my $reader, my $writer, 'child.pl' );

make_filehandle_non_blocking( $writer );

my $long_string = '0123456789' x ($test_size / 10);
printf "Parent: writing long string ( length: %d )\n", length $long_string;
my $sel_writers = IO::Select->new( $writer );
my $sel_readers = IO::Select->new( $reader );
my $read_offset = 0;
my $write_offset = 0;
my $child_output = '';
while (1) {
    last if $sel_readers->count() == 0 && $sel_writers->count() == 0;
    my @sel_result = IO::Select::select( $sel_readers, $sel_writers, undef );
    my @read_ready = @{ $sel_result[0] };
    my @write_ready = @{ $sel_result[1] };
    if ( @write_ready ) {
        my $bytes_written = syswrite $writer, $long_string, WRITE_BUF_SIZE, $write_offset;
        if ( !defined $bytes_written ) {
            die "syswrite failed: $!" if $! != EAGAIN;
            $bytes_written = 0;
        }
        $write_offset += $bytes_written;
        if ( $write_offset >= length $long_string ) {
            $sel_writers->remove( $writer );
            close $writer;
        }
    }
    if ( @read_ready ) {
        my $bytes_read = sysread $reader, $child_output, READ_BUF_SIZE, $read_offset;
        if ( !defined $bytes_read ) {
            die "sysread failed: $!" if $! != EAGAIN;
            $bytes_read = 0;
        }
        elsif ( $bytes_read == 0 ) {
            $sel_readers->remove( $reader );
            close $reader;
        }
        $read_offset += $bytes_read;
    }
}
printf "Parent: Got output with length %d..\n", length $child_output;
say "Parent: Reaping child..";
waitpid( $pid, 0 );
my $child_exit_status = $? >> 8;
say "Parent: Child exited with status: $child_exit_status";

sub make_filehandle_non_blocking {
    my ( $fh ) = @_;

    my $flags = fcntl $fh, F_GETFL, 0
      or die "Couldn't get flags for file handle : $!\n";
    fcntl $fh, F_SETFL, $flags | O_NONBLOCK
      or die "Couldn't set flags for file handle: $!\n";

}

这篇关于由于缓冲导致的死锁.它是如何工作的?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！