c ++系统()提出了ENOMEM [英] c++ system() raises ENOMEM
问题描述
这个问题是 M(not)WE 这个问题。我写了一个重现错误的代码:
#include< cstdlib>
pre>
#include< iostream>
#include< vector>
int * watch_errno = __errno_location();
int main(){
std :: vector< double>一个(7e8,1); //分配一大块内存
std :: cout<< system(NULL)<< std :: endl;
$ b必须用
g ++ -ggdb - std = c ++ 11
(Debian上的g ++ 4.9)。注意
int * watch_errno
仅用于允许gdb监视errno
。
当它在
gdb
下运行时,我得到这个:(gdb)watch * watch_errno
硬件观察点1:* watch_errno
(gdb)r
启动程序:/ tmp / bug
硬件观察点1:* watch_errno
旧值=<不可读>
新值= 0
__static_initialization_and_destruction_0(__initialize_p = 1,__priority = 65535)在bug.cpp:10
10}
(gdb)c
继续。
硬件观察点1:* watch_errno
旧值= 0
新值= 12
0x00007ffff7252421 in do_system(line = line @ entry = 0x7ffff7372168exit 0) at ../sysdeps/posix/system.c:116
116 ../sysdeps/posix/system.c:没有这样的文件或目录。
(gdb)bt
#0 0x00007ffff7252421在do_system(line = line @ entry = 0x7ffff7372168exit 0)at ../sysdeps/posix/system.c:116
#1 0x00007ffff7252510 ()中的__libc_system(line =< out out>)在../sysdeps/posix/system.c:182
#2 0x0000000000400ad8在main()上bug.cpp:9
(gdb)l
111 in ../sysdeps/posix/system.c
(gdb)c
继续。
0
[下1(进程5210)正常退出]
原因
errno
在第9行被设置为ENOMEM
,这对应于
系统()
呼叫。请注意,如果向量具有较小的大小(我猜它
取决于您将运行代码的计算机),代码可以正常工作,并且
system(NULL)
返回1,因为它应该有一个shell可用。
为什么标记
ENOMEM
上调?为什么不使用交换内存的代码?这是一个错误?有没有解决方法?会不会popen
或exec *
做同样的事情? (我知道,我应该只在每个帖子中提出一个问题,但是所有这些问题都可以归纳为正在发生什么?)
按照要求,这里是
ulimit -a
的结果:-t:cpu time(秒)无限
-f:文件大小(块)无限制
-d:数据段大小(千字节)无限制
-s:堆栈大小(千字节)8192
-c:核心文件大小(块)0
-m:驻留集大小(千字节)无限
-u:进程30852
-n:文件描述符65536
-l:locked-in - 内存大小(千字节)64
-v:地址空间(千字节)无限制
-x:文件锁无限制
-i:挂起信号30852
-q:POSIX中的字节msg队列819200
-e:max nice 0
-r:max rt priority 0
-N 15:unlimited
,这里是
strace -f myprog
<$ (SIGINT,{SIG_IGN,[],SA_RESTORER,0x7fabe622b180},pmaprt_sigaction {SIG_DFL,[],0},8)= 0
rt_sigaction(SIGQUIT,{SIG_IGN,[],SA_RESTORER,0x7fabe622b180},{SIG_DFL,[],0},8)= 0
rt_sigprocmask (SIG_BLOCK,[CHLD],[],8)= 0
clone(child_stack = 0,flags = CLONE_PARENT_SETTID | SIGCHLD,parent_tidptr = 0x7fff8797635c)= -1 ENOMEM(无法分配内存)
rt_sigaction ,{SIG_DFL,[],SA_RESTORER,0x7fabe622b180},NULL,8)= 0
rt_sigaction(SIGQUIT,{SIG_DFL,[],SA_RESTORER,0x7fabe622b180},NULL,8)= 0
rt_sigprocmask(SIG_SETMASK (1,{st_mode = S_IFCHR | 0620,st_rdev = makedev(136,1),...})= 0
mmap(NULL,4096,[],NULL,8)= 0
fstat ,PROT_READ | PROT_WRITE,MAP_PRIVATE | MAP_ANONYMOUS,-1,0)= 0x7fabe6fde000
写(1,0,20
) = 2
write(1,8 \ n,28
)= 2
munmap(0x7faa98562000,5600002048)= 0
这里是免费的输出:
共享缓冲区缓存
pre
Mem:7915060 1668928 6246132 49576 34668 1135612
- / + buffers / cache:498648 7416412
Swap:2928636 0 2928636
解决方案
系统()函数首先创建一个新的使用
fork()
或类似的方法复制进程(在Linux中,这将在clone()
系统调用中结束,然后在子进程中调用exec
来创建一个运行所需命令的shell。
<如果新进程的虚拟内存不足,fork()
调用可能会失败(即使您打算立即将其替换为一个小得多的内存,内核不知道)。写入时复制(vfork()
)或内存过度使用()可以减少页面错误可能失败的保证, code> / proc / sys / vm / overcommit_memory 和/ proc / sys / vm / overcommit_ratio
)。
请注意,上述同样适用于任何可能会创建新进程的库函数 - 例如
POPEN()
。虽然不是exec()
,因为取代了过程,并且不克隆它。
如果提供的机制不适合您的用例,那么您可能需要实现自己的
system()
替换。我建议尽早开始一个子进程(在分配大量内存之前),它的唯一工作是接受NUL
-stdin code>并在
stdout
中报告退出状态。
后一种解决方案的大纲看起来像这样:
int request_fd [2];
int reply_fd [2];
pipe(request_fd);
pipe(reply_fd);
if(fork()){
/ * in parent * /
close(request_fd [0]);
close(reply_fd [1]);
} else {
/ * in child * /
close(request_fd [1]);
close(reply_fd [0]);
while(read(request_fd [0],command)){
int result = system(command);
write(reply_fd [1],结果);
}
exit();
}
//重要:不要在fork()之后分配
std :: vector< double>一个(7e8,1); //分配一大块内存
int my_system_replacement(const char * command){
write(request_fd [1],command);
read(reply_fd [0],结果);
返回结果;
}
您需要在整个过程中添加适当的错误检查,页面。你可能希望使它更面向对象,也许使用iostreams来进行读写操作等。
This question is a M(not)WE of this question. I wrote a code that reproduces the error:
#include <cstdlib> #include <iostream> #include <vector> int *watch_errno = __errno_location(); int main(){ std::vector<double> a(7e8,1); // allocate a big chunk of memory std::cout<<system(NULL)<<std::endl; }
It has to be compiled with
g++ -ggdb -std=c++11
(g++ 4.9 on a Debian). Note that theint *watch_errno
is useful only to allow gdb to watcherrno
.When it is run under
gdb
, I get this :(gdb) watch *watch_errno Hardware watchpoint 1: *watch_errno (gdb) r Starting program: /tmp/bug Hardware watchpoint 1: *watch_errno Old value = <unreadable> New value = 0 __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at bug.cpp:10 10 } (gdb) c Continuing. Hardware watchpoint 1: *watch_errno Old value = 0 New value = 12 0x00007ffff7252421 in do_system (line=line@entry=0x7ffff7372168 "exit 0") at ../sysdeps/posix/system.c:116 116 ../sysdeps/posix/system.c: No such file or directory. (gdb) bt #0 0x00007ffff7252421 in do_system (line=line@entry=0x7ffff7372168 "exit 0") at ../sysdeps/posix/system.c:116 #1 0x00007ffff7252510 in __libc_system (line=<optimized out>) at ../sysdeps/posix/system.c:182 #2 0x0000000000400ad8 in main () at bug.cpp:9 (gdb) l 111 in ../sysdeps/posix/system.c (gdb) c Continuing. 0 [Inferior 1 (process 5210) exited normally]
For some reason
errno
is set toENOMEM
at line 9 which corresponds to thesystem()
call. Note that if the vector has a smaller size (I guess that it depends on which computer you'll run the code), the code works fine andsystem(NULL)
returns 1 as it should when a shell is available.Why is the flag
ENOMEM
raised? Why isn't the code using the swap memory? Is this a bug? Is there a workaround? Wouldpopen
orexec*
do the same? (I know, I should only ask one question per post, but all these question could be summarized by, "what is going on?")As requested, here is the result of
ulimit -a
:-t: cpu time (seconds) unlimited -f: file size (blocks) unlimited -d: data seg size (kbytes) unlimited -s: stack size (kbytes) 8192 -c: core file size (blocks) 0 -m: resident set size (kbytes) unlimited -u: processes 30852 -n: file descriptors 65536 -l: locked-in-memory size (kbytes) 64 -v: address space (kbytes) unlimited -x: file locks unlimited -i: pending signals 30852 -q: bytes in POSIX msg queues 819200 -e: max nice 0 -r: max rt priority 0 -N 15: unlimited
and here the relevant part of
strace -f myprog
mmap(NULL, 5600002048, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7faa98562000 rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7fabe622b180}, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7fabe622b180}, {SIG_DFL, [], 0}, 8) = 0 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7fff8797635c) = -1 ENOMEM (Cannot allocate memory) rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fabe622b180}, NULL, 8) = 0 rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fabe622b180}, NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fabe6fde000 write(1, "0\n", 20 ) = 2 write(1, "8\n", 28 ) = 2 munmap(0x7faa98562000, 5600002048) = 0
here is the output of free:
total used free shared buffers cached Mem: 7915060 1668928 6246132 49576 34668 1135612 -/+ buffers/cache: 498648 7416412 Swap: 2928636 0 2928636
解决方案The
system()
function works by first creating a new copy of the process withfork()
or similar (in Linux, this ends up in theclone()
system call, as you show) and then, in the child process, callingexec
to create a shell running the desired command.The
fork()
call can fail if there is insufficient virtual memory for the new process (even though you intend to immediately replace it with a much smaller footprint, the kernel can't know that). Some systems allow you to trade the ability to fork large processes for reduced guarantees that page faults may fail, with copy-on-write (vfork()
) or memory overcommit (/proc/sys/vm/overcommit_memory
and/proc/sys/vm/overcommit_ratio
).Note that the above applies equally to any library function that may create new processes - e.g.
popen()
. Though notexec()
, as that replaces the process and doesn't clone it.If the provided mechanisms are inadequate for your use case, then you may need to implement your own
system()
replacement. I recommend starting a child process early on (before you allocate lots of memory) whose sole job is to acceptNUL
-separated command lines onstdin
and report exit status onstdout
.An outline of the latter solution in pseudo-code looks something like:
int request_fd[2]; int reply_fd[2]; pipe(request_fd); pipe(reply_fd); if (fork()) { /* in parent */ close(request_fd[0]); close(reply_fd[1]); } else { /* in child */ close(request_fd[1]); close(reply_fd[0]); while (read(request_fd[0], command)) { int result = system(command); write(reply_fd[1], result); } exit(); } // Important: don't allocate until after the fork() std::vector<double> a(7e8,1); // allocate a big chunk of memory int my_system_replacement(const char* command) { write(request_fd[1], command); read(reply_fd[0], result); return result; }
You'll want to add appropriate error checks throughout, by reference to the man pages. And you might want to make it more object-oriented, and perhaps use iostreams for your read and write operations, etc.
这篇关于c ++系统()提出了ENOMEM的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文