Ruby:无法分配内存 [英] Ruby: Cannot allocate memory

问题描述

我正在开发 Ruby on Rails 应用程序.我是 Ruby/Rails 的新手.我使用 Ruby 2.2.0 和 Rails 4.2.当我运行如下命令时:

I am in the process of development of a Ruby on Rails application. I am a newbie to Ruby/Rails. I use Ruby 2.2.0 and Rails 4.2. When I run a command like:

rails g migration SomeMigrationName

它失败了

Cannot allocate memory - fork(2) (Errno::ENOMEM)

我使用 2014 年中期的 Macbook Pro 和 OS X 10.10 以及 Vagrant/Virtualbox 来运行虚拟机 (Ubuntu 14.04) 以进行 Rails 开发.

I use Macbook Pro mid 2014 with OS X 10.10 on board and Vagrant/Virtualbox to run a virtual machine (Ubuntu 14.04) for Rails development.

这是我的 Vagrant 文件:

Here is my Vagrant file:

Vagrant.configure(2) do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.network "forwarded_port", guest: 3000, host: 3000
  config.vm.synced_folder "dev", "/home/vagrant/dev"
  config.vm.synced_folder "opt", "/opt"
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "512"
  end
end

我了解到当 RAM 超出限制时会发生此类错误，但我对另一个运行多个 Python/Tornado 应用程序、MongoDB 和 Redis 的开发环境使用相同的配置(Vagrant 文件)，并且一切正常.

I've read that such an error occurs when RAM is out of limit, but I use same config (Vagrant file) for the another dev environment which runs several Python/Tornado apps, MongoDB and Redis and it all works fine.

我需要增加 vb.memory 值还是 Ruby 错误?

Do I need to increase vb.memory value or it's a Ruby bug?

推荐答案

当 Ruby 调用 fork 时，操作系统会复制整个父进程的地址空间，即使 fork 只是被调用exec 另一个类似于 ls 的小进程.暂时，您的系统需要能够分配至少与 Ruby 父进程大小相同的内存块，然后才能将其缩减为子进程实际需要的大小.

When Ruby calls fork the OS will make a copy of the entire parent processes address space, even if fork is only being called to exec another small process like ls. Momentarily, your system needs to be able to allocate a chunk of memory at least the size of the Ruby parent process before collapsing it down to what the child process actually needs.

所以 Rails 通常非常需要内存.然后，如果某些东西使用 fork，你需要两倍的内存.

So rails is generally quite memory hungry. Then if something uses fork, you need twice as much memory.

TL;DR 使用 posix-spawn 而不是 fork 如果您可以控制代码.否则给你的虚拟机 1024MB 或一点额外的交换空间来填补 fork 调用

TL;DR Use posix-spawn instead of fork if you are in control of the code. Otherwise give your VM 1024MB or a bit of extra swap space to take up the slack for the fork call

使用fork

随机抽取一个虚拟机，这个虚拟机禁用了交换空间:

Take a random VM, this one has swap space disabled:

$ free -m
             total       used       free     shared    buffers     cached
Mem:          1009        571        438          0          1         35
-/+ buffers/cache:        534        475
Swap:            0          0          0

查看 Mem: 行和 free 列.这大约是新进程的大小限制，在我的情况下 438MiB

Look at the Mem: row and free column. This is around about your size limit for a new process, in my case 438MiB

我的 buffers/cached 已经为这个测试刷新，以便我的free 内存已达到极限.如果 buffers/cache 值很大，您可能需要考虑它们.当进程需要内存时，Linux 能够清除陈旧的缓存.

My buffers/cached have already been flushed for this test so that my free memory is at it's limit. You may need to take the buffers/cache values into account if they are large. Linux has the ability to evict stale cache when memory is needed by a process.

占用一些内存

创建一个 ruby​​ 进程，其中包含一个与可用内存大小相当的字符串.ruby 进程有一些开销，因此它不会与 free 完全匹配.

Create a ruby process with a string around the size of your free memory. There is some overhead for the ruby process so it's not going to exactly match free.

$ ruby -e 'mb = 380; a="z"*mb*2**20; puts "=)"'
=)

然后将字符串稍微大一点:

Then make the string slightly larger:

$ ruby -e 'mb = 385; a="z"*mb*2**20; puts "=)"'
-e:1:in `*': failed to allocate memory (NoMemoryError)
        from -e:1:in `<main>'

向 ruby​​ 进程添加一个 fork，减少 mb 直到它运行.

Add a fork to the ruby process, reducing mb until it runs.

$ ruby -e 'mb = 195; a="z"*mb*2**20; fork; puts "=)"'
=)

稍大的 fork 进程会产生 ENOMEM 错误:

$ ruby -e 'mb = 200; a="z"*mb*2**20; fork; puts "=)"'
-e:1:in `fork': Cannot allocate memory - fork(2) (Errno::ENOMEM)
        from -e:1:in `<main>'

运行带有反引号的命令会使用 fork 启动该进程，因此结果相同:

Running a command with backticks launches that process with a fork so has the same outcome:

$ ruby -e 'mb = 200; a="z"*mb*2**20; `ls`'
-e:1:in ``': Cannot allocate memory - ls (Errno::ENOMEM)
        from -e:1:in `<main>'

因此，您需要大约两倍于系统上可用的父进程内存来创建一个新进程.MRI Ruby 严重依赖 fork 的多进程模型，这是由于 Ruby 的设计使用了 全局解释器锁 (GIL)，每个 ruby​​ 进程一次只允许一个线程执行.

So there you go, you need about twice the parent processes memory available on the system to fork a new process. MRI Ruby relies heavily on fork for it's multi process model, this is due to the design of Ruby which uses a global interpreter lock (GIL) that only allows one thread to execute at a time per ruby process.

我相信 Python 在内部很少使用 fork.当您在 Python 中使用 os.fork 时，会发生同样的情况:

I believe Python has a lot less use of fork internally. When you do use os.fork in Python, the same occurs though:

python -c 'a="c"*420*2**20;'
python -c 'import os; a="c"*200*2**20; os.fork()'

Oracle 有一篇有关该问题的详细文章 并讨论使用 posix_spawn() 的替代方案.这篇文章是针对 Solaris 的，但这是一个普遍的 POSIX Unix 问题，因此适用于 Linux(如果不是大多数 Unices).

Oracle have a detailed article on the problem and talk about using the alternative of posix_spawn(). The article is directed at Solaris but this is a general POSIX Unix issue so applies to Linux (if not most Unices).

还有一个 posix-spawn 的 Ruby 实现如果您可以控制代码，则可以使用.此模块不会替换 Rails 中的任何内容，因此除非您自己替换对 fork 的调用，否则它不会在这里帮助您.

There is also a Ruby implementation of posix-spawn which you could use if you are in control of the code. This module doesn't replace anything in Rails, so it won't help you here unless you replaced the calls to fork yourself.

这篇关于Ruby:无法分配内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！