c ++系统（）提出了ENOMEM [英] c++ system() raises ENOMEM

问题描述

这个问题是 M（not）WE 这个问题。我写了一个重现错误的代码：

  #include< cstdlib> 
 #include< iostream> 
 #include< vector> 
 
 int * watch_errno = __errno_location（）; 
 
 int main（）{
 std :: vector< double>一个（7e8,1）; //分配一大块内存
 std :: cout<< system（NULL）<< std :: endl; 
 
 
 
 $ b 
必须用 g ++ -ggdb  - std = c ++ 11 （Debian上的g ++ 4.9）。注意
  int * watch_errno 仅用于允许gdb监视 errno 。  
 
 
当它在 gdb 下运行时，我得到这个：
 （gdb）watch * watch_errno 
硬件观察点1：* watch_errno 
（gdb）r 
启动程序：/ tmp / bug 
硬件观察点1：* watch_errno 
 
旧值=<不可读> 
新值= 0 
 __static_initialization_and_destruction_0（__initialize_p = 1，__priority = 65535）在bug.cpp：10 
 10} 
（gdb）c 
继续。 
硬件观察点1：* watch_errno 
 
旧值= 0 
新值= 12 
 0x00007ffff7252421 in do_system（line = line @ entry = 0x7ffff7372168exit 0） at ../sysdeps/posix/system.c:116 
 116 ../sysdeps/posix/system.c：没有这样的文件或目录。 
（gdb）bt 
＃0 0x00007ffff7252421在do_system（line = line @ entry = 0x7ffff7372168exit 0）at ../sysdeps/posix/system.c:116 
＃1 0x00007ffff7252510 （）中的__libc_system（line =< out out>）在../sysdeps/posix/system.c:182 
＃2 0x0000000000400ad8在main（）上bug.cpp：9 
（gdb）l 
 111 in ../sysdeps/posix/system.c 
（gdb）c 
继续。 
 0 
 [下1（进程5210）正常退出] 
  
原因 errno 在第9行被设置为 ENOMEM ，这对应于
 系统（）呼叫。请注意，如果向量具有较小的大小（我猜它
取决于您将运行代码的计算机），代码可以正常工作，并且
  system（NULL）返回1，因为它应该有一个shell可用。
 
 
 
为什么标记 ENOMEM 上调？为什么不使用交换内存的代码？这是一个错误？有没有解决方法？会不会 popen 或 exec * 做同样的事情？ （我知道，我应该只在每个帖子中提出一个问题，但是所有这些问题都可以归纳为正在发生什么？） 
 
 
按照要求，这里是 ulimit -a 的结果：
  -t：cpu time（秒）无限
 -f：文件大小（块）无限制
 -d：数据段大小（千字节）无限制
 -s：堆栈大小（千字节）8192 
 -c：核心文件大小（块）0 
 -m：驻留集大小（千字节）无限
 -u：进程30852 
 -n：文件描述符65536 
 -l：locked-in - 内存大小（千字节）64 
 -v：地址空间（千字节）无限制
 -x：文件锁无限制
 -i：挂起信号30852 
 -q：POSIX中的字节msg队列819200 
 -e：max nice 0 
 -r：max rt priority 0 
 -N 15：unlimited 
  
，这里是 strace -f myprog  
 
 
 <$ （SIGINT，{SIG_IGN，[]，SA_RESTORER，0x7fabe622b180}，pmap  rt_sigaction {SIG_DFL，[]，0}，8）= 0 
 rt_sigaction（SIGQUIT，{SIG_IGN，[]，SA_RESTORER，0x7fabe622b180}，{SIG_DFL，[]，0}，8）= 0 
 rt_sigprocmask （SIG_BLOCK，[CHLD]，[]，8）= 0 
 clone（child_stack = 0，flags = CLONE_PARENT_SETTID | SIGCHLD，parent_tidptr = 0x7fff8797635c）= -1 ENOMEM（无法分配内存）
 rt_sigaction ，{SIG_DFL，[]，SA_RESTORER，0x7fabe622b180}，NULL，8）= 0 
 rt_sigaction（SIGQUIT，{SIG_DFL，[]，SA_RESTORER，0x7fabe622b180}，NULL，8）= 0 
 rt_sigprocmask（SIG_SETMASK （1，{st_mode = S_IFCHR | 0620，st_rdev = makedev（136，1），...}）= 0 
 mmap（NULL，4096，[]，NULL，8）= 0 
 fstat ，PROT_READ | PROT_WRITE，MAP_PRIVATE | MAP_ANONYMOUS，-1,0）= 0x7fabe6fde000 
写（1，0，20 
） = 2 
 write（1，8 \ n，28 
）= 2 
 munmap（0x7faa98562000,5600002048）= 0 
  pre> 
 
 
这里是免费的输出： 
 
 
 共享缓冲区缓存
 Mem：7915060 1668928 6246132 49576 34668 1135612 
  -  / + buffers / cache：498648 7416412 
 Swap：2928636 0 2928636 
  pre 
 
解决方案 
  
系统（）函数首先创建一个新的使用 fork（）或类似的方法复制进程（在Linux中，这将在 clone（）系统调用中结束，然后在子进程中调用 exec 来创建一个运行所需命令的shell。
 
 
 <如果新进程的虚拟内存不足， fork（）调用可能会失败（即使您打算立即将其替换为一个小得多的内存，内核不知道）。写入时复制（ vfork（））或内存过度使用（）可以减少页面错误可能失败的保证， code> / proc / sys / vm / overcommit_memory 和 / proc / sys / vm / overcommit_ratio ）。
 
 
 
请注意，上述同样适用于任何可能会创建新进程的库函数 - 例如 POPEN（）。虽然不是 exec（），因为取代了过程，并且不克隆它。
 
 
 
如果提供的机制不适合您的用例，那么您可能需要实现自己的 system（）替换。我建议尽早开始一个子进程（在分配大量内存之前），它的唯一工作是接受 NUL   -   stdin  code>并在 stdout 中报告退出状态。
 
 
 
后一种解决方案的大纲看起来像这样：
  int request_fd [2]; 
 int reply_fd [2]; 
 
 pipe（request_fd）; 
 pipe（reply_fd）; 
 
 if（fork（））{
 / * in parent * / 
 close（request_fd [0]）; 
 close（reply_fd [1]）; 
} else {
 / * in child * / 
 close（request_fd [1]）; 
 close（reply_fd [0]）; 
 while（read（request_fd [0]，command））{
 int result = system（command）; 
 write（reply_fd [1]，结果）; 
} 
 exit（）; 
} 
 
 //重要：不要在fork（）之后分配
 std :: vector< double>一个（7e8,1）; //分配一大块内存
 
 int my_system_replacement（const char * command）{
 write（request_fd [1]，command）; 
 read（reply_fd [0]，结果）; 
返回结果; 
} 
  
您需要在整个过程中添加适当的错误检查，页面。你可能希望使它更面向对象，也许使用iostreams来进行读写操作等。
 
This question is a M(not)WE of this question. I wrote a code that reproduces the error:
#include <cstdlib>
#include <iostream>
#include <vector>

int *watch_errno = __errno_location();

int main(){
    std::vector<double> a(7e8,1);  // allocate a big chunk of memory
    std::cout<<system(NULL)<<std::endl;
}
It has to be compiled with g++ -ggdb -std=c++11 (g++ 4.9 on a Debian). Note
that the int *watch_errno is useful only to allow gdb to watch errno. 

When it is run under gdb, I get this :
(gdb) watch *watch_errno 
Hardware watchpoint 1: *watch_errno
(gdb) r
Starting program: /tmp/bug 
Hardware watchpoint 1: *watch_errno

Old value = <unreadable>
New value = 0
__static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at bug.cpp:10
10      }
(gdb) c
Continuing.
Hardware watchpoint 1: *watch_errno

Old value = 0
New value = 12
0x00007ffff7252421 in do_system (line=line@entry=0x7ffff7372168 "exit 0") at ../sysdeps/posix/system.c:116
116     ../sysdeps/posix/system.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7252421 in do_system (line=line@entry=0x7ffff7372168 "exit 0") at ../sysdeps/posix/system.c:116
#1  0x00007ffff7252510 in __libc_system (line=<optimized out>) at ../sysdeps/posix/system.c:182
#2  0x0000000000400ad8 in main () at bug.cpp:9
(gdb) l
111     in ../sysdeps/posix/system.c
(gdb) c
Continuing.
0
[Inferior 1 (process 5210) exited normally]
For some reason errno is set to ENOMEM at line 9 which corresponds to the
system() call. Note that if the vector has a smaller size (I guess that it
depends on which computer you'll run the code), the code works fine and
system(NULL) returns 1 as it should when a shell is available.

Why is the flag ENOMEM raised?  Why isn't the code using the swap memory? Is this a bug?  Is there a workaround?  Would popen or exec* do the same? (I know, I should only ask one question per post, but all these question could be summarized by, "what is going on?")

As requested, here is the result of ulimit -a:
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-m: resident set size (kbytes)      unlimited
-u: processes                       30852
-n: file descriptors                65536
-l: locked-in-memory size (kbytes)  64
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 30852
-q: bytes in POSIX msg queues       819200
-e: max nice                        0
-r: max rt priority                 0
-N 15:                              unlimited
and here the relevant part of strace -f myprog
mmap(NULL, 5600002048, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7faa98562000
rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7fabe622b180}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7fabe622b180}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7fff8797635c) = -1 ENOMEM (Cannot allocate memory)
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fabe622b180}, NULL, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fabe622b180}, NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fabe6fde000
write(1, "0\n", 20
)                      = 2
write(1, "8\n", 28
)                      = 2
munmap(0x7faa98562000, 5600002048)      = 0
here is the output of free: 
           total       used       free     shared    buffers     cached
Mem:       7915060    1668928    6246132      49576      34668    1135612
-/+ buffers/cache:     498648    7416412
Swap:      2928636          0    2928636

解决方案
The system() function works by first creating a new copy of the process with fork() or similar (in Linux, this ends up in the clone() system call, as you show) and then, in the child process, calling exec to create a shell running the desired command.

The fork() call can fail if there is insufficient virtual memory for the new process (even though you intend to immediately replace it with a much smaller footprint, the kernel can't know that).  Some systems allow you to trade the ability to fork large processes for reduced guarantees that page faults may fail, with copy-on-write (vfork()) or memory overcommit (/proc/sys/vm/overcommit_memory and /proc/sys/vm/overcommit_ratio).

Note that the above applies equally to any library function that may create new processes - e.g. popen().  Though not exec(), as that replaces the process and doesn't clone it.

If the provided mechanisms are inadequate for your use case, then you may need to implement your own system() replacement.  I recommend starting a child process early on (before you allocate lots of memory) whose sole job is to accept NUL-separated command lines on stdin and report exit status on stdout.

An outline of the latter solution in pseudo-code looks something like:
int request_fd[2];
int reply_fd[2];

pipe(request_fd);
pipe(reply_fd);

if (fork()) {
    /* in parent */
    close(request_fd[0]);
    close(reply_fd[1]);
} else {
    /* in child */
    close(request_fd[1]);
    close(reply_fd[0]);
    while (read(request_fd[0], command)) {
        int result = system(command);
        write(reply_fd[1], result);
    }
    exit();
}

// Important: don't allocate until after the fork()
std::vector<double> a(7e8,1);  // allocate a big chunk of memory

int my_system_replacement(const char* command) {
    write(request_fd[1], command);
    read(reply_fd[0], result);
    return result;
}
You'll want to add appropriate error checks throughout, by reference to the man pages.  And you might want to make it more object-oriented, and perhaps use iostreams for your read and write operations, etc.

                        
这篇关于c ++系统（）提出了ENOMEM的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！