尝试打开4个以上命名管道（FIFO）后，Node.js fs.open（）挂起 [英] Node.js fs.open() hangs after trying to open more than 4 named pipes (FIFOs)
问题描述
我有一个node.js进程需要从不同的其他进程中读取多个命名管道作为IPC方法。
I have a node.js process that needs to read from multiple named pipes fed by different other processes as an IPC method.
我在打开并创建读取后意识到来自超过四个十五的流，fs似乎不再能够打开fifos而只是挂在那里。
I realized after opening and creating read streams from more than four fifos, that fs seems to no longer be able to open fifos and just hangs there.
考虑到可以同时打开数千个文件而不会出现问题（例如更换 mkfifo），这个数字似乎有点低通过以下脚本中的触摸。
It seems that this number is a bit low, considering that it is possible to open thousands of files concurrently without trouble (for instance by replacing mkfifo by touch in the following script).
我使用node.js v10进行了测试Mac OS 10.13上的.1.0和Ubuntu 16.04上的node.js v8.9.3，结果相同。
I tested with node.js v10.1.0 on MacOS 10.13 and with node.js v8.9.3 on Ubuntu 16.04 with the same result.
错误的脚本
以及显示此行为的脚本：
And a script that displays this behavior:
var fs = require("fs");
var net = require("net");
var child_process = require('child_process');
var uuid = function() {
for (var i = 0, str = ""; i < 32; i++) {
var number = Math.floor(Math.random() * 16);
str += number.toString(16);
}
return str;
}
function setupNamedPipe(cb) {
var id = uuid();
var fifoPath = "/tmp/tmpfifo/" + id;
child_process.exec("mkfifo " + fifoPath, function(error, stdout, stderr) {
if (error) {
return;
}
fs.open(fifoPath, 'r+', function(error, fd) {
if (error) {
return;
}
var stream = fs.createReadStream(null, {
fd
});
stream.on('data', function(data) {
console.log("FIFO data", data.toString());
});
stream.on("close", function(){
console.log("close");
});
stream.on("error", function(error){
console.log("error", error);
});
console.log("OK");
cb();
});
});
}
var i = 0;
function loop() {
++i;
console.log("Open ", i);
setupNamedPipe(loop);
}
child_process.exec("mkdir -p /tmp/tmpfifo/", function(error, stdout, stderr) {
if (error) {
return;
}
loop();
});
此脚本不会在他身后清理，不要忘记 rm -r / tmp / tmpfifo
This script doesn't clean behind him, don't forget to rm -r /tmp/tmpfifo
注意，以下部分问题与我已经尝试回答这个问题，但可能不是它的核心
这有两个有趣的事实脚本
Two interesting facts with this script
- 在其中一个FIFO中写入两次时，（即
echo hello> fifo）当通过直接提供到fifo的路径创建读取流时，节点可以再打开一个fifo，但不再从我们编写的那个中接收 - （而不是fd），脚本不再阻止，但显然不再接收任何FIFO中写的内容
- when writing twice in one of the FIFO, (ie
echo hello > fifo) Node is then able to open one more fifo, but no longer receives from the one in which we wrote - when the read stream is created by directly providing the path to the fifo (instead of fd), the script doesn't block any more, but apparently no longer receive what is written in any of the FIFOs
调试信息
然后我尝试验证这是否与某些操作系统限制有关，例如文件描述符的打开数量。
I then tried to verify whether that could be related to some OS limit, for instance the number of file descriptor open.
输出<$ Mac上的c $ c> ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1418
virtual memory (kbytes, -v) unlimited
4点没有任何限制。
C ++暂定
然后我尝试在C ++中编写类似的脚本。
在C ++中，脚本成功打开了一百个。
I then tried to write a similar script in C++. In C++ the script successfully open a hundred fifos.
请注意，这两个实现之间存在一些差异。在C ++中，
Note that there are a few differences between the two implementations. In the C++ one,
- 脚本只打开fifos，
- 没有暂定的阅读，
- 且没有多线程
#include <string>
#include <cstring>
#include <sys/stat.h>
#include <fcntl.h>
#include <iostream>
int main(int argc, char** argv)
{
for (int i=0; i < 100; i++){
std::string filePath = "/tmp/tmpfifo/" + std::to_string(i);
auto hehe = open(filePath.c_str(), O_RDWR);
std::cout << filePath << " " << hehe << std::endl;
}
return 0;
}
作为旁注，需要在执行脚本之前创建fifos，例如，
As a side note, the fifos need to be created before executing the script, for instance with
for $ in $（seq 0 100）;做mkfifo / tmp / tmpfifo / $ i;完成
潜在的Node.js相关问题
经过一些搜索，它似乎也与Node.js Github上的那个问题有关：
After a bit of search, it also seems to be linked to that issue on the Node.js Github:
< a href =https://github.com/nodejs/node/issues/1941\"rel =nofollow noreferrer> https://github.com/nodejs/node/issues/1941 。
但人们似乎在抱怨相反的行为（fs.open（）抛出EMFILE错误而不是默默地悬挂......）
But people seems to be complaining of the opposite behavior (fs.open() throwing EMFILE errors and not hanging silently...)
如你所见，我试图在很多方向上进行搜索，这一切都引出了我的问题：
As you can see I tried to search in many directions and all of this lead me to my question:
你知道什么会导致这种行为吗？
谢谢你
推荐答案
所以我在Node.js Github上提出问题， https ：//github.com/nodejs/node/issues/23220
So I asked the question on the Node.js Github, https://github.com/nodejs/node/issues/23220
来自解决方案：
处理FIFO目前有点棘手。
Dealing with FIFOs is currently a bit tricky.
open（）默认情况下在FIFO上的系统调用块，直到管道的另一侧也被打开。因为Node.js使用线程池进行文件系统操作，所以打开多个管道，其中 open（）调用没有完成耗尽这个线程池。
The open() system call blocks on FIFOs by default until the other side of the pipe has been opened as well. Because Node.js uses a threadpool for file-system operations, opening multiple pipes where the open() calls don’t finish exhausts this threadpool.
解决方案是以非阻塞模式打开文件，但是其他 fs 调用的问题不是用非 - 记住文件描述符; net.Socket 然而。
The solution is to open the file in non-blocking mode, but that has the difficulty that the other fs calls aren’t built with non-blocking file descriptors in mind; net.Socket is, however.
所以，解决方案看起来像这样：
So, the solution would look something like this:
fs.open('path/to/fifo/', fs.constants.O_RDONLY | fs.constants.O_NONBLOCK, (err, fd) => {
// Handle err
const pipe = new net.Socket({ fd });
// Now `pipe` is a stream that can be used for reading from the FIFO.
});
这篇关于尝试打开4个以上命名管道（FIFO）后，Node.js fs.open（）挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
