为什么缺少缺少的块时，读取系统调用会停止读取? [英] Why read system call stops reading when less than block is missing?

问题描述

我正在尝试从子进程(通过从父级调用popen生成)发送图像到父级进程.

I am trying to send an image from a child process (generated by calling popen from the parent) to the parent process.

图像是灰度png图像.它使用OpenCV库打开，并使用同一库的imencode函数进行编码.因此，将所得的编码数据存储到类型为uchar的std::vector结构中，即下面的代码中的buf向量.

The image is a grayscale png image. It is opened with the OpenCV library and encoded using imencode function of the same library. So the resulting encoded data is stored into a std::vector structure of type uchar, namely the buf vector in the code below.

首先，孩子发送父母需要的以下图像信息:

First the child sends the following image information needed by the parent:

包含编码数据的buf向量的

• 大小:需要此信息，以便父​​级将分配相同大小的缓冲区，以将其将从子级接收到的图像信息写入其中.分配如下执行(在这种情况下，buf是用于接收数据的数组，而不是包含编码数据的向量):

• size of the buf vector containing the encoded data: this piece of information is needed so that the parent will allocate a buffer of the same size where to write the image information that it will receive from the child. Allocation is performed as follows (buf in this case is the array used to received data not the vector containing the encoded data):

u_char *buf = (u_char*)malloc(val*sizeof(u_char));

原始图像的行数:在接收到所有数据之后，父级需要对图像进行解码；

原始图像的列数:在接收到所有数据之后，父级需要对图像进行解码.

这些数据由子级使用cout写入标准输出，并由父级使用fgets系统调用读取.

These data are written by the child on the standard output using cout and read by the parent using fgets system call.

这些信息已正确发送和接收，因此到目前为止没有问题.

This pieces of information are correctly sent and received so no problem until now.

子级使用write系统调用将编码数据(即向量buf中包含的数据)写入标准输出，而父级则使用popen返回的文件描述符读取数据.使用read系统调用读取数据.

The child writes the encoded data (i.e. the data contained in the vector buf) to the standard output using write system call while the parent uses the file-descriptor returned by popen to read the data. Data is read using read system call.

数据写入和读取在while循环内以4096个字节的块进行.编写行如下:

Data writing and reading is performed in blocks of 4096 bytes inside while loops. The writing line is the following:

written += write(STDOUT_FILENO, buf.data()+written, s);

其中，STDOUT_FILENO指示在标准输出上写. buf.data()将指针返回到向量结构内部使用的数组中第一个元素的指针. written存储到现在为止已写入的字节数，并用作索引. s是write每次尝试发送的字节数(4096). write返回实际已写入的字节数，该字节数用于更新written.

where STDOUT_FILENO tells to write on standard output. buf.data() returns the pointer to the first element in the array used internally by the vector structure. written stores the number of bytes that have been written until now and it is used as index. s is the number of bytes (4096) that write will try to send each time. write returns the number of bytes that actually have been written and this is used to update written.

数据读取非常相似，它由以下行完成:

Data reading is very similar and it is performed by the following line:

bytes_read = read(fileno(fp), buf+total_bytes, bytes2Copy);

fileno(fp)告诉从哪里读取数据(fp是popen返回的文件描述符). buf是存储接收到的数据的数组，而total_bytes是直到现在为止读取的字节数，因此将其用作索引. bytes2Copy是预期要接收的字节数:BUFLEN(即4096)凋零，或者最后一个数据块剩余的数据(例如，如果总字节为5000，则在1个块之后) 4096个字节的数据，则需要另一个5000-4096块.)

fileno(fp) is telling from where to read data (fp is the filedescriptor returned by popen). buf is the array where received data is stored and total_bytes are the number of bytes read until now so it is used as index. bytes2Copy is the number of bytes expected to be received: it is wither BUFLEN (i.e. 4096) or for the last block of data the remaining data (if for example the total bytes are 5000 then after 1 block of 4096 bytes another block of 5000-4096 is expected).

请考虑以下示例.以下是使用popen

Consider this example. The following is a process launching a child process with popen

#include <stdlib.h>
#include <unistd.h>//read
#include "opencv2/opencv.hpp"
#include <iostream>
#define BUFLEN 4096

int main(int argc, char *argv[])
{
    //file descriptor to the child process
    FILE *fp;
    cv::Mat frame;
    char temp[10];
    size_t bytes_read_tihs_loop = 0;
    size_t total_bytes_read = 0;
    //launch the child process with popen
    if ((fp = popen("/path/to/child", "r")) == NULL)
    {
        //error
        return 1;
    }

    //read the number of btyes of encoded image data
    fgets(temp, 10, fp);
    //convert the string to int
    size_t bytesToRead = atoi((char*)temp);

    //allocate memory where to store encoded iamge data that will be received
    u_char *buf = (u_char*)malloc(bytesToRead*sizeof(u_char));

    //some prints
    std::cout<<bytesToRead<<std::endl;

    //initialize the number of bytes read to 0
    bytes_read_tihs_loop=0;
    int bytes2Copy;
    printf ("bytesToRead: %ld\n",bytesToRead);
    bytes2Copy = BUFLEN;
    while(total_bytes_read<bytesToRead &&
        (bytes_read_tihs_loop = read(fileno(fp), buf+total_bytes_read, bytes2Copy))
    )
    {
        //bytes to be read at this iteration: either 4096 or the remaining (bytesToRead-total)
        bytes2Copy = BUFLEN < (bytesToRead-total_bytes_read) ? BUFLEN : (bytesToRead-total_bytes_read);
        printf("%d btytes to copy\n", bytes2Copy);
        //read the bytes
        printf("%ld bytes read\n", bytes_read_tihs_loop);

        //update the number of bytes read
        total_bytes_read += bytes_read_tihs_loop;
        printf("%lu total bytes read\n\n", total_bytes_read);
    }
    printf("%lu bytes received over %lu expected\n", total_bytes_read, bytesToRead);
    printf("%lu final bytes read\n", total_bytes_read);
    pclose(fp);
    cv::namedWindow( "win", cv::WINDOW_AUTOSIZE );
    frame  = cv::imdecode(cv::Mat(1,total_bytes_read,0, buf), 0);
    cv::imshow("win", frame);

    return 0;

}

和上面打开的过程对应于以下内容:

and the process opened by the above corresponds to the following:

#include <unistd.h> //STDOUT_FILENO
#include "opencv2/opencv.hpp"
#include <iostream>
using namespace std;
using namespace cv;

#define BUFLEN 4096

int main(int argc, char *argv[])
{
    Mat frame;
    std::vector<uchar> buf;
    //read image as grayscale
    frame = imread("test.png",0);
    //encode image and put data into the vector buf
    imencode(".png",frame, buf);
    //send the total size of vector to parent
    cout<<buf.size()<<endl;
    unsigned int written= 0;

    int i = 0;
    size_t toWrite = 0;
    //send until all bytes have been sent
    while (written<buf.size())
    {
        //send the current block of data
        toWrite = BUFLEN < (buf.size()-written) ? BUFLEN : (buf.size()-written);
        written += write(STDOUT_FILENO, buf.data()+written, toWrite);
        i++;
    }
    return 0;

}

错误

孩子读取图像，对其进行编码，然后首先将尺寸(大小，#行，#cols)发送给父对象，然后再发送编码后的图像数据.

The error

The child reads an image, encodes it and sends first the dimensions (size, #rows, #cols) to the parent and then the encoded image data.

父级首先读取尺寸(没有尺寸问题)，然后开始读取数据.每次迭代读取数据4096个字节.但是，当缺少少于4096个字节时，它将尝试仅读取缺少的字节:在我的情况下，最后一步应该读取1027个字节(115715%4096)，但是与其读取所有字节，不如直接读取`15

The parent reads first the dimensions (no prob with that), then it starts reading data. Data is read 4096 bytes at each iteration. However when less than 4096 bytes are missing, it tries to read only the missing bytes: in my case the last step should read 1027 bytes (115715%4096), but instead of reading all of them it just reads `15.

我在前两次迭代中打印的是:

What I got printed for the last two iterations is:

4096 btytes to copy
1034 bytes read
111626 total bytes read

111626 bytes received over 115715 expected
111626 final bytes read
OpenCV(4.0.0-pre) Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /path/window.cpp, line 356
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.0.0-pre) /path/window.cpp:356: error: (-215:Assertion failed) size.width>0 && size.height>0 in function 'imshow'

Aborted (core dumped)

为什么read不读取所有丢失的字节?

Why isn't read reading all the missing bytes?

我正在处理此图像:

在我尝试对图像进行解码的过程中也可能会出现错误，因此也将不胜感激.

There might be errors also on how I am trying to decode back the image so any help there would be appreciated too.

编辑

在我看来，与某些建议相反，该问题与\n或\r或\0的存在无关.

In my opinion as opposed to some suggestions the problem is not related to the presence of \n or \r or \0.

实际上，当我用以下几行打印接收为整数的数据时:

In fact when I print data received as integer with the following lines:

for (int ii=0; ii<val; ii++)
{
    std::cout<<(int)buf[ii]<< " ";
}

我在数据中间看到了0，10和13值(上述字符的ASCII值)，所以我认为这不是问题.

I see 0, 10 and 13 values (the ASCII values of the above mentioned characters) in the middle of data so this makes me think it is not the problem.

推荐答案

fgets(temp, 10, fp);
...
read(fileno(fp), ...)

这可能行不通.

stdio例程是缓冲的.缓冲区由实现控制. fgets(temp, 10, fp);将从文件中读取未知数目的字节并将其放入缓冲区.这些字节将再也不会被低级文件IO看到.

stdio routines are buffered. Buffers are controlled by the implementation. fgets(temp, 10, fp); will read an unknown number of bytes from the file and put it in a buffer. These bytes will never be seen by low level file IO again.

您永远都不会在两种IO样式中都使用相同的文件.使用stdio可以执行任何操作，或者使用低级IO可以执行任何操作.到目前为止，第一个选项是最简单的，只需将read替换为fread.

You never, ever, use the same file with both styles of IO. Either do everything with stdio, or do everything with low-level IO. The first option is the easiest by far, you just replace read with fread.

如果出于某种邪恶的原因而只知道黑暗的邪恶力量，而您想要保留两种IO风格，则可以在执行其他任何操作之前通过调用setvbuf(fp, NULL, _IOLBF, 0)进行尝试.我从来没有这样做过，不能保证使用这种方法，但是他们说它应该起作用.我看不出有任何理由使用它.

If for some ungodly reason known only to the evil forces of darkness you want to keep both styles of IO, you can try that by calling setvbuf(fp, NULL, _IOLBF, 0) before doing anything else. I have never done that and cannot vouch for this method, but they say it should work. I don't see a single reason to use it though.

在可能不相关的地方，请注意，您的阅读循环在终止条件中具有一些逻辑，不太容易理解并且可能无效.读取文件的正常方式大致如下:

On a possibly unrelated, note, your reading loop has some logic in its termination condition that is not so easy to understand and could be invalid. The normal way to read a file looks approximately as follows:

 left = data_size;
 total = 0;
 while (left > 0 &&
        (got=read(file, buf+total, min(chunk_size, left))) > 0) {
    left -= got;
    total += got;
 }

 if (got == 0) ... // reached the end of file
 else if (got < 0) ... // encountered an error

---

更正确的方法是got < 0 && errno == EINTR，然后重试，因此修改后的条件看起来像

---

The more correct way would be to try again if got < 0 && errno == EINTR, so the modified condition could look like

 while (left > 0 &&
        (((got=read(file, buf+total, min(chunk_size, left))) > 0) ||
        (got < 0 && errno == EINTR))) {

但是这时可读性开始受到损害，您可能希望将其拆分为单独的语句.

but at this point readability starts to suffer and you may want to split this in separate statements.

这篇关于为什么缺少缺少的块时，读取系统调用会停止读取?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！