耙任务内存泄漏 [英] rake task memory leak

问题描述

我有一个长时间运行的 rake 任务，随着时间的推移它会吞噬我所有的系统内存?追踪我的问题并深入了解问题的最快方法是什么?

I have a long running rake task that is gobbling up all my system memory over time? What is the quickest way to track down my issue and get to the bottom of it?

我在 slicehost 和 mysql 5 上使用 rails 2.3.5、ruby 1.8.7、ubuntu.

Im using rails 2.3.5, ruby 1.8.7, ubuntu on slicehost and mysql 5.

我有运行良好的 rails 应用程序.我有一个整夜运行的夜间工作，做了大量的工作(一些外部调用 twitter、google 等，以及大量使用活动记录的数据库调用，随着时间的推移，该工作的内存大小增加到近 4 演出.我需要弄清楚找出为什么 rake 任务没有释放内存.

I have rails app that works fine. I have a nightly job that runs all night and does tons of work (some external calls to twitter, google etc, and lots of db calls using active record, over time that job grows in memory size to nearly 4 gig. I need to figure out why the rake task is not releasing memeory.

我开始研究 bleak_house，但它的设置似乎很复杂，而且已经一年多没有更新了.我无法让它在本地工作，所以我不愿意在生产中尝试.

I started looking into bleak_house, but it seems complex to setup and hasnt been updated in over a year. I cant get it to work locally so im reluctant to try in production.

谢谢乔尔

推荐答案

抛出两个想法.首先，如果您将循环作为这项工作的一部分，请确保您没有保留对不需要的对象的引用，因为这会阻止它们被收集.如果你完成了，从你的阵列中删除它们，或者其他什么.此外，将定期 GC.start 放入您的循环中，以查看它是否根本没有绕过 GC.

Throwing out two ideas. First, if you're looping as part of this job, make sure you're not holding onto references to objects you don't need, as this will prevent them from being collected. If you're done, remove them from your array, or whatever. Also, put a periodic GC.start into your loop as a way to see if it's simply not getting around to GC-ing.

第二个想法是 ruby​​ 不会 GC 符号，所以如果你的 API 客户端将值存储为符号，你最终可能会得到一个巨大且不断增长的符号集，这些符号永远不会被重用.符号很小，但微小的东西仍然可以加起来.

Second idea is that ruby does not GC symbols, so if your API clients are storing values as symbols you can end up with a huge and growing set of symbols that will never be re-used. Symbols are tiny, but tiny things can still add up.

当然，不要加载过多的对象.如果您必须迭代大量 AR 对象，请使用 #find_each 批量加载它们.

And of course, don't load more objects than you need to. use #find_each to load AR objects in batches if you have to iterate over lots of them.

这篇关于耙任务内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！