Node.js“致命错误:JS分配失败 - 进程内存不足"-- 可以获得堆栈跟踪吗? [英] Node.js "FATAL ERROR: JS Allocation failed - process out of memory" -- possible to get a stack trace?

问题描述

嗯……我又回到了第一个.我一辈子都搞不清楚.

Well... I'm back to square one. I can't figure this out for the life of me.

我收到以下错误:

FATAL ERROR: JS Allocation failed - process out of memory

我可以列举出很多(是的，几十个)我试图找到这个问题根源的事情，但真的太多了.所以这里是关键点:

I could enumerate the dozens (yes, dozens) of things I've tried to get to the root of this problem, but really it would be far too much. So here are the key points:

• 我只能在我的生产服务器上让它发生，而且我的应用又大又复杂，所以很难隔离
• 即使堆大小 & 也会发生RSS 大小都是 <200 Mb，考虑到机器(Amazon Cloud、CentOS、m1.large)具有 8Gb RAM，这应该不是问题

我的假设是(因为第二点)，泄漏可能不是原因；相反，似乎有一个非常大的 SINGLE 对象.以下线程支持这一理论:在 Node.js 中使用 JSON.stringify 导致进程内存不足"错误

My assumption is that (because of the 2nd point), a leak is probably not the cause; rather, it seems like there's probably a SINGLE object that is very large. The following thread backs up this theory:: In Node.js using JSON.stringify results in 'process out of memory' error

我真正需要的是某种方法来找出应用程序崩溃时内存的状态，或者可能是导致致命错误的堆栈跟踪.

What I really need is some way to find out what the state of the memory is at the moment the application crashes, or perhaps a stack trace leading up to the FATAL ERROR.

根据我上面的假设，10 分钟前的堆转储是不够的(因为对象不会驻留在内存中).

Based upon my assumption above, a 10-minute-old heap dump is insufficient (since the object would have not resided in memory).

推荐答案

我必须在这个问题上为 Trevor Norris 提供巨大的支持用于帮助修改 node.js 本身，以便在发生此错误时自动生成堆转储.

I have to give huge props to Trevor Norris on this one for helping to modify node.js itself such that it would automatically generate a heap dump when this error happened.

不过，最终为我解决这个问题的方法要平凡得多.我编写了一些简单的代码，将每个传入 API 请求的端点附加到日志文件中.我等待收集约 10 个数据点(崩溃)并比较崩溃前 60 秒运行的端点.我发现在 9/10 的案例中，一个端点在崩溃前刚刚被击中.

Ultimately what solved this problem for me, though, was much more mundane. I wrote some simple code that appended the endpoint of each incoming API request to a log file. I waited to gather ~10 data points (crashes) and compared the endpoints which had been run 60sec before the crash. I found that in 9/10 cases, a single endpoint that had been hit just before the crash.

从那里开始，只需深入挖掘代码即可.我削减了一切——从我的 mongoDB 查询中返回更少的数据，只将必要的数据从对象传回回调，等等.现在我们已经比平均时间长了 6 倍，没有任何服务器发生一次崩溃，这让我希望现在已经解决了……

From there, it was just a matter of digging deeper into the code. I pared everything down -- returning less data from my mongoDB queries, passing only necessary data from an object back to the callback, etc. Now we've gone 6x longer than average without a single crash on any of the servers, leading me to hope that it is resolved... for now.

这篇关于Node.js“致命错误:JS分配失败 - 进程内存不足"-- 可以获得堆栈跟踪吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！