on Rails的内存泄漏时，通过大量的记录循环红宝石; find_each没有帮助 [英] Ruby on Rails memory leak when looping through large number of records; find_each doesn't help

问题描述

我有一个Rails应用程序，处理在MySQL数据库中记录的大（百万）数量。一旦开始工作，其内存占用快速增长以每秒50MB的速度。有了这样OINK工具，我是能够缩小的根本原因一个循环，经过​​在一个大的数据库表中的所有记录。

I have a Rails app that processes a large (millions) number of records in a mysql database. Once it starts working, its memory use quickly grows at a speed of 50MB per second. With tools like oink I was able to narrow down the root cause to one loop that goes through all the records in a big table in the database.

我明白，如果我使用类似的 Person.all.each ，所有的记录都将被加载到内存中。但是，如果我切换到 find_each ，我仍看到相同的内存问题。为了进一步隔离问题，我创建了下面的测试控制器，它什么都不做，而是通过记录循环。我猜想 find_each 的只保留一小部分在每个时间内存中的对象，但内存使用线性增长，因为它执行。

I understand if I use something like Person.all.each, all the records will be loaded into memory. However if I switch to find_each, I still see the same memory issue. To further isolate the problem I created the following test controller, which does nothing but looping through the records. I suppose find_each only keeps a small number of objects in memory each time, but memory use grows linearly as it executes.

class TestController < ApplicationController
  def memory_test
    Person.find_each do |person|
    end
end

我怀疑它必须与ActiveRecord的缓存查询结果。但我检查我的环境设置，我有（我使用的由轨道创建的默认设置）设置为false发展的所有缓存相关的选项。我做了一些网上搜索，但没有找到一个解决方案。

I suspect it has to do with ActiveRecord caching the query results. But I checked my environment settings and I do have all the caching related options set to false in development (I am using the default settings created by rails). I did some search online but couldn't find a solution.

我使用的铁轨3.1.0 RC1和Ruby 1.9.2

I am using rails 3.1.0 rc1 and ruby 1.9.2

谢谢！

推荐答案

我能算出这个我自己。有两个地方，以改变

I was able to figure this out myself. There are two places to change.

首先，禁止IdentityMap。在配置/ application.rb中

First, disable IdentityMap. In config/application.rb

config.active_record.identity_map = false

二，使用未缓存的收官之环

class MemoryTestController < ApplicationController
  def go
    ActiveRecord::Base.uncached do
      Person.find_each do |person|
        # whatever operation
      end
    end
  end
end

现在我的记忆中使用已受到控制。希望这可以帮助其他人。

Now my memory use is under control. Hope this helps other people.

这篇关于on Rails的内存泄漏时，通过大量的记录循环红宝石; find_each没有帮助的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！