V8隔离映射的内存泄漏 [英] V8 isolates mapped memory leaks

问题描述

需要V8开发人员.

我注意到以下代码泄漏了映射内存(mmap，munmap)，具体来说，cat /proc/<pid>/maps中的映射区域数量持续增长并很快达到系统限制(/proc/sys/vm/max_map_count).

I've noticed that the following code leaks mapped memory (mmap, munmap), concretely the amount of mapped regions within cat /proc/<pid>/maps continuously grows and hits the system limit pretty quickly (/proc/sys/vm/max_map_count).

void f() {
  auto platform = v8::platform::CreateDefaultPlatform();
  v8::Isolate::CreateParams create_params;

  create_params.array_buffer_allocator =
    v8::ArrayBuffer::Allocator::NewDefaultAllocator();

  v8::V8::InitializePlatform(platform);
  v8::V8::Initialize();

  for (;;) {
    std::shared_ptr<v8::Isolate> isolate(v8::Isolate::New(create_params), [](v8::Isolate* i){ i->Dispose(); });
  }

  v8::V8::Dispose();
  v8::V8::ShutdownPlatform();

  delete platform;
  delete create_params.array_buffer_allocator;
}

我在platform-linux.cc文件中玩了一点，发现UncommitRegion调用只是用PROT_NONE重新映射了区域，但没有释放它.也许多数民众赞成在某种程度上与该问题有关.

I've played a little bit with platform-linux.cc file and have found that UncommitRegion call just remaps region with PROT_NONE, but not release it. Probably thats somehow related to that problem..

我们在程序执行期间重新创建隔离的原因有很多.

There are several reasons why we recreate isolates during the program execution.

第一个是从GC角度来看，创建新的隔离株以及丢弃旧的隔离株更可预测.基本上，我发现

The first one is that creating new isolate along with discarding the old one is more predictable in terms of GC. Basically, I found that doing

auto remoteOldIsolate = std::async(
    std::launch::async,
    [](decltype(this->_isolate) isolateToRemove) { isolateToRemove->Dispose(); },
    this->_isolate
);

this->_isolate = v8::Isolate::New(cce::Isolate::_createParams);

// 

与调用LowMemoryNotification相比，

更容易预测和更快.因此，我们使用GetHeapStatistics监视内存消耗，并在达到极限时重新创建隔离.原来，我们不能将GC活动视为代码执行的一部分，这会导致不良的用户体验.

is more predictable and faster than call to LowMemoryNotification. So we monitor memory consumptions using GetHeapStatistics and recreate isolate when it hits the limit. Turns out we cannot consider GC activity as a part of code execution, this leads to bad user experience.

第二个原因是每个代码都有隔离符，这样可以并行运行多个代码，否则v8::Locker会阻止该特定隔离符的第二个代码.

The second reason is that having isolate per code allows as to run several codes in parallel, otherwise v8::Locker will block second code for that particular isolate.

看起来在这个阶段我别无选择，并且将重写应用程序以使每个代码具有隔离池和持久性上下文..当然，这种方式code#1可能会通过进行许多分配而影响code#2，并且GC将在code2完全没有分配，但至少不会泄漏.

Looks like at this stage I have no choices and will rewrite application to have a pool of isolates and persistent context per code..of course this way code#1 may affect code#2 by doing many allocations and GC will run on code2 with no allocations at all, but at least it will not leak.

PS.我已经提到我们使用GetHeapStatistics进行内存监视.我想澄清一下那部分.

PS. I've mentioned that we use GetHeapStatistics for memory monitoring. I want to clarify a little bit that part.

在我们的案例中，当GC在代码执行期间工作时，这是一个大问题.每个代码都有执行超时时间(100-500ms).在代码执行过程中进行GC活动会锁定代码，有时我们有时会因分配操作而超时. GC回调无法为您提供足够的准确性，因此我们不能依赖它们.

In our case its a big problem when GC works during code execution. Each code has execution timeout (100-500ms). Having GC activity during code execution locks code and sometimes we have timeouts just for assignment operation. GC callbacks don't give you enough accuracy, so we cannot rely on them.

实际上，我们指定--max-old-space-size=32000(32GB).这样，GC不想运行，因为它应该看到存在大量内存.使用GetHeapStatistics(以及上面提到的隔离娱乐)，我们可以进行手动内存监视.

What we actually do, we specify --max-old-space-size=32000 (32GB). That way GC don't want to run, cuz it should see that a lot of memory exists. And using GetHeapStatistics (along with isolate recreation I've mentioned above) we have manual memory monitoring.

PPS.我还提到过，在代码之间共享隔离可能会影响用户. 假设您有user#1和user#2.他们每个人都有自己的代码，两者都不相关. code#1有一个具有大量内存分配的循环，code#2只是一个分配操作. GC可能会在code#2期间运行，而user#2会收到超时.

PPS. I also mentioned that sharing isolate between codes may affect users. Say you have user#1 and user#2. Each of them have their own code, both are unrelated. code#1 has a loop with tremendous memory allocation, code#2 is just an assignment operation. Chances are GC will run during code#2 and user#2 will receive timeout.

推荐答案

需要V8开发人员.

V8 developer is needed.

请在 crbug.com/v8/new 中提交错误.请注意，此问题可能被认为是低优先级；我们通常会假设每个进程的隔离数量仍然保持合理(即不成千上万).

Please file a bug at crbug.com/v8/new. Note that this issue will probably be considered low priority; we generally assume that the number of Isolates per process remains reasonably small (i.e., not thousands or millions).

有很多隔离区

have a pool of isolates

是的，这可能就是要走的路.特别是，正如您已经写过的，如果要并行执行脚本，则每个线程需要一个隔离.

Yes, that's probably the way to go. In particular, as you already wrote, you will need one Isolate per thread if you want to execute scripts in parallel.

这样，代码#1可能会通过执行许多分配来影响代码#2，而GC将在完全没有分配的代码2上运行

this way code#1 may affect code#2 by doing many allocations and GC will run on code2 with no allocations at all

不，那不会发生.只有分配会触发GC活动.免分配代码将花费零时间进行GC.另外(正如我们之前在前面的问题中所讨论的那样)，GC活动被分为许多微小的步骤(通常是亚毫秒级的步骤)(这些步骤又是由分配触发的)，因此特别是短时间运行的代码不会遇到一些巨大的GC暂停.

No, that can't happen. Only allocations trigger GC activity. Allocation-free code will spend zero time doing GC. Also (as we discussed before in your earlier question), GC activity is split into many tiny (typically sub-millisecond) steps (which in turn are triggered by allocations), so in particular a short-running bit of code will not encounter some huge GC pause.

有时我们仅出于赋值操作而超时

sometimes we have timeouts just for assignment operation

这听起来令人惊讶，而且听起来与GC不相关；我敢打赌，还有其他事情正在发生，但是我不确定那可能是什么.你有生殖器吗?

That sounds surprising, and doesn't sound GC-related; I would bet that something else is going on, but I don't have a guess as to what that might be. Do you have a repro?

我们指定--max-old-space-size = 32000(32GB).这样，GC不想运行，因为它应该看到存在大量内存.使用GetHeapStatistics(以及上面提到的隔离娱乐)，我们可以进行手动内存监视.

we specify --max-old-space-size=32000 (32GB). That way GC don't want to run, cuz it should see that a lot of memory exists. And using GetHeapStatistics (along with isolate recreation I've mentioned above) we have manual memory monitoring.

您是否尝试过不进行任何此类操作?默认情况下，V8的GC进行了非常精细的调整，我认为以这种方式回避它会导致超出其解决范围的问题.当然，您可以尝试任何您喜欢的东西.但是如果产生的行为不是您想要的，那么我的第一个建议就是让V8做到这一点，并且只有在您发现默认行为不能令人满意的情况下进行干预.

Have you tried not doing any of that? V8's GC is very finely tuned by default, and I would assume that side-stepping it in this way is causing more problems than it solves. Of course you can experiment with whatever you like; but if the resulting behavior isn't what you were hoping for, then my first suggestion is to just let V8 do its thing, and only interfere if you find that the default behavior is somehow unsatisfactory.

code#1有一个具有大量内存分配的循环，code#2只是一个赋值操作. GC可能会在代码2中运行，而用户2将收到超时.

code#1 has a loop with tremendous memory allocation, code#2 is just an assignment operation. Chances are GC will run during code#2 and user#2 will receive timeout.

再次:不.未分配的代码不会被GC中断.而且，同一隔离中的多个功能永远无法并行运行；一个隔离中只能同时有一个线程处于活动状态.

Again: no. Code that doesn't allocate will not be interrupted by GC. And several functions in the same Isolate can never run in parallel; only one thread may be active in one Isolate at the same time.

这篇关于V8隔离映射的内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！