Promise 重试模式内存占用 [英] Promise Retry Pattern Memory Footprint

问题描述

考虑一个发出 HTTP 请求直到成功的函数:

Consider a function to make a HTTP request until it succeeds:

function fetch_retry(url) {
    return fetch(url).catch(function(error) {
        return fetch_retry(url);
    });
}

随着失败次数的增加，内存消耗是否增加?

As the number of failure increases, does the memory consumption increase?

我猜调用堆栈的高度是 O(1) 但我不知道是否保留了 clojure 上下文(随时间增长).

I guess the callstack height is O(1) but I don't know if the clojure context (growing over time) is retained.

function fetch_retry(url) {
    return fetch(url).catch(function(error) {
      //return fetch_retry(url)+1;
        return fetch_retry(url).then(res=>{res.body+='a'});
    });
}

(假设 fetch 解析为一个数字)常量 1 应该在内存中，因为它稍后会用到.

(assume that fetch resolves to a number) The constant 1 should be in memory because it is used later.

function fetch_retry(url, odd_even) {
    return fetch(url).catch(function(error) {
      //return fetch_retry(url, 1-odd_even)+odd_even;
        return fetch_retry(url, 1-odd_even).then(res=>{res.body+=str(odd_even)});
    });
}

fetch_retry('http://..', 0)

通过交替 0 和 1，编译器无法重用操作数.

By alternating 0 and 1, the compiler cannot reuse the operand.

没有人解释案例2和案例3，所以我进行了实验.令人惊讶的是，所有场景中的内存都增加了.

No one explained about the case2 and 3, so I run the experiment. Surprisingly the memory increases in all scenarios.

function f(n) {
    return new Promise(function (resolve, reject) {
        gc();
        if(n==10000) return resolve(0);
        // else return f(n+1).then(value=>value+n);
        // else return f(n+1).then(value=>value);
        else return resolve(f(n+1));
    });
}

f(0).then(res => {
    console.log('result: ', res);
    const used = process.memoryUsage().heapUsed / 1024 / 1024;
    console.log(`memory: ${Math.round(used * 100) / 100} MB`);
}).finally(()=>console.log('finished'));

并运行

node --stack_size=999999 --expose_gc test.js

请注意，我每次都运行 gc() 以防止延迟 GC.

Note that I ran gc() every time to prevent delayed GC.

• n=1000，内存:4.34 MB
• n=10000，内存:8.95 MB

9000 个堆栈为 5MB = 每次调用 590 字节.

5MB for 9000 stacks = 590 bytes per call.

一种可能是保留每个栈中的resolve函数.要删除它，

One possibility is that resolve function in each stack is retained. To remove this,

function f(n) {
    return new Promise(function (resolve, reject) {
        gc();
        if(n==10000) resolve(0);
        // else return f(n+1).then(value=>value+n);
        else return f(n+1).then(value=>value);
        // else return f(n+1);

        const used = process.memoryUsage().heapUsed / 1024 / 1024;
        console.log(`memory: ${Math.round(used * 100) / 100} MB`);
    });
}
f(0);

• n=1000，内存:4.12 MB
• n=10000，内存:7.07 MB

所以堆栈是平的，但内存并不像我们想象的那样干净?

So the stack is flat but the memory is not clean as we think?

推荐答案

随着失败次数的增加，内存消耗是否增加?

As the number of failure increases, does the memory consumption increase?

它没有.Promises 的现代实现没有它们起源的 Promise 的记忆.我可以验证在 Firefox、Chrome、Safari 和 Edge 中也是如此.

It does not. Modern implementations of Promises have no memory of promises they were originated from. I can verify that this is the case in Firefox, Chrome, Safari and Edge.

'旧' 实现有这个问题(想想 Q).现代浏览器和像 bluebird 这样的快速库会在解析后分离"引用.

'old' implementations had this problem (think Q). Modern browsers and fast libraries like bluebird 'detach' the reference after resolving.

也就是说，现在没有理由递归实现这个，你可以这样做:

That said, there is no reason to implement this recursively now, you can just do:

async function fetch_retry(url) {
  while(true) { // or limit retries somehow
    try {
      await fetch(url);
    } catch { }
  }
}

请注意，永远重试是一个坏主意，我强烈建议对后端更友好，并在重试和限制重试次数时使用指数退避.

Note that retrying forever is a bad idea, I warmly recommend being friendlier to the backend and using exponential backoff when making retries and limiting the amount of retries.

这实际上很难从你的角度衡量，因为当调试器附加到 V8(Chrome 的 JavaScript 引擎)中时，异步堆栈跟踪被收集，并且在你的递归示例内存中将增长.

This will actually be hard to measure from your end since when the debugger is attached in V8 (Chrome's JavaScript engine) async stack traces are collected and in your recursive example memory will grow.

您可以从 Chrome 开发工具中关闭异步堆栈跟踪(有一个复选框).

You can turn off async stack traces from the Chrome devtools (there is a checkbox).

这会在开发中泄漏一点(堆栈帧)，但在生产中不会.

This will leak a little bit (the stack frames) in development but not in production.

如果 fetch 本身在错误时泄漏内存(它以前曾为某些类型的请求做过)，那么代码显然会泄漏.据我所知，所有已修复的常青浏览器都发现了漏洞，但未来可能会有漏洞.

If fetch itself leaks memory on errors (which it has done before at times for certain types of requests) then the code will obviously leak. As far as I know all evergreen browsers fixed found leaks but there might be one in the future.

这篇关于Promise 重试模式内存占用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！