NodeJS内存增长-(系统)内存泄漏? [英] NodeJS memory growth - Memory leak in (system)?

问题描述

我在生活环境中遇到了一个奇怪的内存泄漏，堆中的(系统)对象不断增长.

I am encountering a strange memory leak in our live-environment, where (system) objects in the heap keep growing.

这是一个内存转储，内存使用量增加到800MB:

Here is a memory dump where the memory usage grew to 800MB:

请注意，此内存保留在 Generator 对象中.TBH，我不知道那是什么.

Notice that this memory is retained in a Generator object. TBH, I have no idea what that is.

通过 global.gc(); 手动触发垃圾回收通常会释放大约10MB的内存.(但是重复触发gc没有影响)

Manually triggering garbage collection via global.gc(); usally frees about 10MB of memory. (But repetitive triggering of gc has no affect)

应用程序大约每12小时崩溃一次，并显示以下错误:

The application crashes roughly every 12 hours with following error:

2019-11-15T08:29:01.174417825Z 08:29:01 0|. | <--- Last few GCs --->
2019-11-15T08:29:01.176095829Z 08:29:01 0|. | [47:0x55ac8cdf3dc0] 65890519 ms: Scavenge 912.3 (929.2) -> 909.3 (929.2) MB, 6.7 / 0.0 ms  (average mu = 0.987, current mu = 0.987) allocation failure 
2019-11-15T08:29:01.177260332Z 08:29:01 0|. | [47:0x55ac8cdf3dc0] 65890653 ms: Scavenge 912.9 (929.2) -> 910.0 (929.2) MB, 10.1 / 0.0 ms  (average mu = 0.987, current mu = 0.987) allocation failure 
2019-11-15T08:29:01.178027234Z 08:29:01 0|. | [47:0x55ac8cdf3dc0] 65890701 ms: Scavenge 913.6 (929.2) -> 910.6 (929.2) MB, 6.3 / 0.0 ms  (average mu = 0.987, current mu = 0.987) allocation failure 
2019-11-15T08:29:01.183406747Z 08:29:01 0|. | <--- JS stacktrace --->
2019-11-15T08:29:01.184194849Z 08:29:01 0|. | ==== JS stack trace =========================================
2019-11-15T08:29:01.184939851Z 08:29:01 0|. |     0: ExitFrame [pc: 0x55ac88379c39]
2019-11-15T08:29:01.185674553Z 08:29:01 0|. | Security context: 0x064ed4d408d1 <JSObject>
2019-11-15T08:29:01.187183757Z 08:29:01 0|. |     1: /* anonymous */ [0x34eb4a55a171] [/usr/src/app/node_modules/express-validator/src/chain/context-runner-impl.js:~25] [pc=0x34ab58ee2e1f](this=0x3c92e4142b89 <ContextRunnerImpl map = 0x13b2881d9399>)
2019-11-15T08:29:01.188904261Z 08:29:01 0|. |     2: next [0x64ed4d63621](this=0x34eb4a55a569 <JSGenerator>)
2019-11-15T08:29:01.190831866Z 08:29:01 0|. |     3: /* anonymous */(aka /* anonymous */) [0x34eb4a55a201] [/usr/src/app/node_modules/expr...
2019-11-15T08:29:01.195055876Z 08:29:01 0|. | FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory
2019-11-15T08:29:01.396743877Z 2019-11-15T08:29:01: PM2 log: App [.:0] exited with code [0] via signal [SIGABRT]
2019-11-15T08:29:01.400481986Z 08:29:01 PM2      | App [.:0] exited with code [0] via signal [SIGABRT]
2019-11-15T08:29:01.403095193Z 2019-11-15T08:29:01: PM2 log: App [.:0] starting in -fork mode-
2019-11-15T08:29:01.406294501Z 08:29:01 PM2      | App [.:0] starting in -fork mode-
2019-11-15T08:29:01.419367433Z 2019-11-15T08:29:01: PM2 log: App [.:0] online
2019-11-15T08:29:01.422309441Z 08:29:01 PM2      | App [.:0] online

2019-11-15T08:29:03.279070252Z 08:29:03 0|.      | server started at http://localhost:80

大约发生1.2 GB的进程内存使用情况(此时，计算机以约67％的内存运行).这很奇怪，因为我使用以下命令运行服务器:

Which happens around 1.2 GB process memory usage (at which point the machine is running on ~67% memory). This is strange on its own since i run the sever with the following command:

pm2 start . --no-daemon --node-args=\"--max-old-space-size=2048 --max-semi-space-size=4 --expose_gc\" --max-memory-restart 1536M

因此节点应该有2GB的可用空间，并且pm2应该在进程达到限制之前重新启动.

so node should have 2GB available and pm2 should restart well before the process reaches the limit.

该应用程序是用TypeScript编写的json api.它使用 express 处理请求(每分钟约3k)，并使用 node-mssql 查询MS SQL Server.另外，它通过 bee-queue 包异步处理作业(执行附加的sql查询).

The app is a json api written in TypeScript. It uses express to handle requests (about 3k per minute) and uses node-mssql to query an MS SQL Server. Additionally it processes jobs (which perform additional sql queries) asynchronously via the bee-queue package.

推荐答案

因此，我通过编译为 es2017 而不是 es6 解决了该问题.

So, I resolved the issue by compiling to es2017 instead of es6.

我注意到当我将 async 函数传递给 express 路由器时，会发生此问题.但是，返回 Promise 的函数没有问题.

I noticed the issue occurs when i pass async functions to the express router. Yet, functions returning a Promise had no issue.

我发现问题是，当将TypeScript代码编译为 es6 时，它会转换

I found the problem was that when compiled TypeScript code to es6 it converts

async function() {...

到

__awaiter(this, void 0, void 0, function* () {...

这是 Generator 发挥作用的地方.看来 express 在处理请求后无法正确释放它.由于 es2017 支持 async 功能，因此这不再是问题.

This is where Generators come in play. It looks like express cannot correctly free this after handling a request. Since es2017 supports async function this is not a problem anymore.

另外，我还必须删除软件包 express-validator ，因为他们的验证中间件已编译为相同的 Generator 结构.

Additionally I had to remove the package express-validator, because their validation middleware was compiled to the same Generator structure.

这篇关于NodeJS内存增长-(系统)内存泄漏?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！