Erlang：内存使用量数据的差异 [英] Erlang: discrepancy of memory usage figures

问题描述

当我运行我的WebSocket测试，我发现以下有趣的内存使用结果：

服务器声明，没有连接

  [{total，573263528}，
 {processes，17375688}，
 {processes_used，17360240}，
 {system，555887840} 
 {atom，472297}，
 {atom_used，451576}，
 {binary，28944}，
 {code，3774097}，
 {ets，271016}] 
 44进程
系统：705M，
 Erlang居住地：519M 
  

100K连接

  [{total，762564512}，
 {processes，130105104}，
 { process_used，130089656}，
 {system，632459408}，
 {atom，476337}，
 {atom_used，456484}，
 {binary，50160}，
 {代码，3925064}，
 {ets，7589160}] 
 100044进程，
系统：1814M，
 Erlang居住地：950M 
  / pre> 
 
 
 200K连接
 
 
 
（重新启动服务器并从0连接创建，不从案例2继续） / p> 
 
 
  [{total，95204 0232}，
 {processes，243161192}，
 {processes_used，243139984}，
 {system，708879040}，
 {atom，476337}，
 {atom_used， 456484}，
 {二进制，70856}，
 {代码，3925064}，
 {ets，14904760}] 
 200044进程，
系统：3383M，
 Erlang：1837M 
  
带有System：和Erlang：的数字提供htop，其他的是从erlang shell输出的memory（）调用。请看总共和erlang的居留记忆。当没有连接时，这两个大致相同，有100K连接，驻留内存比总数稍大，连接200K，驻留内存几乎是总共的两倍。 
 
 
 
有人可以解释吗？
解决方案
分区是内存碎片。
 
 
 
分配操作系统内存是昂贵的，所以Erlang试图为你管理内存。 
当Erlang分配内存时，它会创建一个名为carrier的实体，该实体由许多块组成。 Erlang内存（total）报告所有块大小（实际使用的内存）的总和。 OS报告所有载波大小（使用的内存和预分配的总和）的总和。可以从Erlang VM读取块大小和载波大小的总和。如果（块尺寸）/（载体大小）< 1，比起虚拟机难以释放运营商。可能有许多大型运营商只使用了几个块。您可以使用以下命令读取它：erlang：system_info（{allocator，Type}）。但有一个更简单的方法。您可以使用Recon库检查：
 
 
 
  http://ferd.github.io/recon/recon_alloc.html  
 
 
 
首先检查：
  recon_alloc：片段化（电流）。 
  
和下一个：
  recon_alloc：碎片（最大值）。 
  
这应该解释Erlang VM和OS报告的总内存之间的差异。如果您通过Websockets发送许多小邮件，您可以通过运行Erlang以2个选项来减少碎片：
  erl + MBas aobf + MBlmbcs 512 
  
第一个选项将会将块分配策略从最合适的角度更改为地址顺序最适合，可以帮助挤压更多的块到第一个运营商，第二个减少最大的多载波载波大小，这使得运营商更小（这应该使其更容易释放）。
 
When I run my WebSocket test, I found the following interesting memory usage results:

Server stated, no connection
[{total,573263528},
 {processes,17375688},
 {processes_used,17360240},
 {system,555887840},
 {atom,472297},
 {atom_used,451576},
 {binary,28944},
 {code,3774097},
 {ets,271016}]
44 processes,
System:705M, 
Erlang Residence:519M
100K Connections
[{total,762564512},
 {processes,130105104},
 {processes_used,130089656},
 {system,632459408},
 {atom,476337},
 {atom_used,456484},
 {binary,50160},
 {code,3925064},
 {ets,7589160}]
100044 processes,
System: 1814M, 
Erlang Residence: 950M
200K Connections 

( restart server and create from 0 connection, not continue from case 2)
[{total,952040232},
 {processes,243161192},
 {processes_used,243139984},
 {system,708879040},
 {atom,476337},
 {atom_used,456484},
 {binary,70856},
 {code,3925064},
 {ets,14904760}]
200044 processes,
System:3383M, 
Erlang: 1837M
The figures with "System:" and "Erlang:" are provided htop, others are output of memory() call from erlang shell. Please look at the total and erlang residence memory. When there is no connection, these two are roughly same, with 100K connections, residence memory is a little larger than total, with 200K connections, residence memory is almost double the total. 

Can anybody explain?
解决方案
The most probable answer for your quersion is memory fragmentation.

Allocating OS memory is expensive, so Erlang tries to manage memory for you.
When Erlang allocates memory, it creates an entity called "carrier", which consists of many "blocks". Erlang memory(total) reports the sum of all the block sizes (memory actually used). OS reports the sum of all carriers sizes (sum of memory used and preallocated). Both sum of blocks sizes and carrier sizes can be read from Erlang VM. If (block sizes)/(carrier sizes) << 1, than VM has hard time with freeing the carriers. There might be many big carriers with only couple of blocks used. You can read it with: erlang:system_info({allocator,Type}). but there is an easier way. You can check it using Recon library:

http://ferd.github.io/recon/recon_alloc.html

Firstly check:
recon_alloc:fragmentation(current).
and next:
recon_alloc:fragmentation(max).
This should explain the difference between total memory reported by Erlang VM and OS. If you are sending many small messages over websockets, you can decrease the fragmentation by running Erlang with 2 options:
erl +MBas aobf +MBlmbcs 512
First option will change the block allocation strategy from best fit to address order best fit, which could help squeeze more blocks into first carriers and second one decreases maximum multiblock carrier size, which makes carriers smaller (this should make freeing them easier).

                        
这篇关于Erlang：内存使用量数据的差异的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！