启动子过程中的竞争状况导致从管道读取到挂起 [英] Race condition in starting up sub processes causes reading from pipe to hang
问题描述
我有两个线程分别启动一个子进程.第一个应用程序是运行很长时间的二进制文件.第二个很快退出.
I have two threads that start up a child process each. The first application is a binary that runs quite long. The second exits quite quickly.
有一个竞争条件，有时会导致失败.下面有一个最低限度的可行代码示例.
There is a race condition that sometimes causes this to fail. Below I have a minimum viable code example.
它使用Boost Process 0.5，它使用标准的fork/execve/dup2系统.关于Boost Process的工作方式有一些技巧，但总的来说效果很好.
It uses Boost Process 0.5, which uses the standard fork / execve / dup2 system. There are some hacks regarding how Boost Process works, but in general it works quite well.
父进程启动了更多的进程，总的来说，它是可行的.
The parent process starts up a lot more processes, and in general it works.
我无法一次启动一个进程，尤其是因为我不知道哪些部分不能交错.
I can't readily boot processes one at a time, especially since I don't know which parts can't interleave.
关于为什么会挂起的任何想法?
Any ideas on why this would hang?
预期输出:
/etc/init.d/led restart: Creating child
Creating child1
Reading STDOUT
/etc/init.d/led restart: Waiting for it to exit
Reading std_err_pipe
wait_for_exit(pullapp);
Reading std_out_pipe
< file list>
Done
但是，它经常但不总是在std_err_pipe处停止.
However, often, but not always, it stops at std_err_pipe.
#include <iostream>
#include <string>
#include <vector>
#include <boost/iostreams/stream.hpp>
#include <boost/process.hpp>
#include <boost/thread.hpp>
void run_sleep()
{
int exit_code;
std::string str;
std::vector< std::string > args;
boost::shared_ptr<boost::process::child> child;
args.push_back(boost::process::search_path("sleep"));
args.push_back("20");
boost::iostreams::stream< boost::iostreams::file_descriptor_source >
out_stream;
boost::process::pipe out_pipe = boost::process::create_pipe();
{
//MUST BE IN SEPARATE SCOPE SO SINK AND SOURCE ARE DESTROYED
// See http://stackoverflow.com/a/12469478/5151127
boost::iostreams::file_descriptor_sink out_sink
(out_pipe.sink, boost::iostreams::close_handle);
boost::iostreams::file_descriptor_source out_source
(out_pipe.source, boost::iostreams::close_handle);
std::cout << "Creating child1" << std::endl;
child.reset(new boost::process::child(
boost::process::execute(
boost::process::initializers::run_exe(args[0]),
boost::process::initializers::set_args(args),
boost::process::initializers::bind_stdout(out_sink),
boost::process::initializers::bind_stderr(out_sink)
)
));
out_stream.open(out_source);
}
std::cout << "Reading STDOUT" << std::endl;
while( out_stream ) {
std::string line;
std::getline(out_stream, line);
std::cout << line << std::endl;
}
std::cout << "wait_for_exit(pullapp);" << std::endl;
exit_code = wait_for_exit(*child);
child.reset();
return;
}
void run_ls()
{
int exit_code;
std::string str;
std::vector< std::string > args ;
args.push_back(boost::process::search_path("ls"));
args.push_back("/lib");
boost::process::pipe std_out_pipe = boost::process::create_pipe();
boost::process::pipe std_err_pipe = boost::process::create_pipe();
std::cout << "/etc/init.d/led restart: Creating child" << std::endl;
{
boost::process::child child = boost::process::execute(
boost::process::initializers::set_args(args),
boost::process::initializers::bind_stdout(
boost::iostreams::file_descriptor_sink(
std_out_pipe.sink,
boost::iostreams::close_handle
)
),
boost::process::initializers::bind_stderr(
boost::iostreams::file_descriptor_sink(
std_err_pipe.sink,
boost::iostreams::close_handle
)
),
boost::process::initializers::throw_on_error()
);
std::cout << "/etc/init.d/led restart: Waiting for it to exit" << std::endl;
exit_code = wait_for_exit(child);
}
{ //with std_err_stream, istream
boost::iostreams::stream< boost::iostreams::file_descriptor_source >
std_err_stream(
boost::iostreams::file_descriptor_source(
std_err_pipe.source, boost::iostreams::close_handle
)
);
std::cout << "Reading std_err_pipe" << std::endl;
std::istream istream(std_err_stream.rdbuf());
while( istream ) {
getline(istream, str);
std::cout << str << std::endl;
}
}
{ //with std_out_stream, istream
boost::iostreams::stream< boost::iostreams::file_descriptor_source >
std_out_stream(
boost::iostreams::file_descriptor_source(
std_out_pipe.source, boost::iostreams::close_handle
)
);
std::cout << "Reading std_out_pipe" << std::endl;
std::istream istream(std_out_stream.rdbuf());
while( istream ) {
getline(istream, str);
std::cout << str << std::endl;
}
}
std::cout << "Done" << std::endl;
}
int main()
{
boost::thread run_sleep_tr(run_sleep);
boost::thread run_ls_tr(run_ls);
run_sleep_tr.join();
run_ls_tr.join();
return 0;
}
(另存为process-test.cpp并使用g++ process-test.cpp -o process-test -lboost_iostreams -lboost_filesystem -lboost_thread -lboost_system进行编译)
(Save as process-test.cpp and compile with g++ process-test.cpp -o process-test -lboost_iostreams -lboost_filesystem -lboost_thread -lboost_system)
推荐答案
显然，这是因为文件句柄最终出现在多个进程中.这些进程不会关闭这些句柄，因此父进程仍在等待.
Apparently this is because file handles end up in multiple processes. Those processes don't close those handles, so the parent remains waiting.
对于Linux，修复相对容易；管道应使用create_pipe中的O_CLOEXEC创建. bind_*方法中的dup2调用清除了此标志，足以使管道正常工作.
For Linux the fix is relatively easy; the pipe should be created with O_CLOEXEC in create_pipe. The dup2 call in the bind_* methods clears this flag, which is enough for the pipe to work properly.
在Windows上，我还没有真正找到解决方案.您必须将句柄标记为可继承的.可能可以在executor()方法中执行此操作，但是这可能需要全局互斥量.我还没有时间适当地研究它.
On Windows, I haven't really found a solution yet. You have to mark the handle as inheritable. It may be possible to do this in the executor() method, but maybe this requires a global mutex. I haven't had the time to properly look into it yet.
这篇关于启动子过程中的竞争状况导致从管道读取到挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
