Erlang内存碎片

Question

我们的Erlang服务器看起来有严重的内存泄漏，VM内存使用率很低，但顶部很高。

环境

Erlang: R16B02 OS: Ubuntu 12.04.5 LTS n l X86_64

Erlang VM

> erlang:memory().
[{total,424544992},
 {processes,293961840},
 {processes_used,293937232},
 {system,130583152},
 {atom,553569},
 {atom_used,521929},
 {binary,9794704},
 {code,14041920},
 {ets,5632280}]

已分配内存

> recon_alloc:memory(allocated).
2570059776
 recon_alloc:memory(allocated_types).
[{binary_alloc,163577856},
 {driver_alloc,11010048},
 {eheap_alloc,2165309440},
 {ets_alloc,11010048},
 {fix_alloc,50855936},
 {ll_alloc,156237824},
 {sl_alloc,2097152},
 {std_alloc,6815744},
 {temp_alloc,3145728}]

分配二进制

> recon:bin_leak(5).
[{<0.440.0>,-769,
  [{current_function,{gen_server,loop,6}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.446.0>,-230,
  [{current_function,{gen_server,loop,6}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.450.0>,-179,
  [{current_function,{gen_server,loop,6}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.12497.0>,-147,
  [{current_function,{gen,do_call,4}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.434.0>,-145,
  [{current_function,{cberl_worker,mget,4}},
   {initial_call,{proc_lib,init_p,5}}]}]

做垃圾

7> erlang:garbage_collect().
true
8> erlang:memory().
[{total,381782256},
 {processes,251371752},
 {processes_used,251361352},
 {system,130410504},
 {atom,553569},
 {atom_used,521929},
 {binary,9230384},
 {code,14041920},
 {ets,5675528}]
> recon_alloc:memory(allocated_types).
[{binary_alloc,150994944},
 {driver_alloc,11010048},
 {eheap_alloc,2154823680},
 {ets_alloc,11010048},
 {fix_alloc,50855936},
 {ll_alloc,156237824},
 {sl_alloc,2097152},
 {std_alloc,6815744},
 {temp_alloc,3145728}]

碎片化

我检查了recon_alloc:fragmentation(current)和recon_alloc:fragmentation(max)，发现一些分配器的当前使用率低于最大使用率。

当前使用情况

 {{binary_alloc,0},
  [{sbcs_usage,1.0},
   {mbcs_usage,0.037804497612847224},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,178384},
   {mbcs_carriers_size,4718592}]},
 {{binary_alloc,2},
  [{sbcs_usage,2.0},
   {mbcs_usage,0.05326200786389803},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,4775112},
   {mbcs_carriers_size,89653248}]},
 {{binary_alloc,1},
  [{sbcs_usage,2.0},
   {mbcs_usage,0.0643930146188447},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,4456384},
   {mbcs_carriers_size,69206016}]},

最大使用量

 {{binary_alloc,0},
  [{sbcs_usage,1.0},
   {mbcs_usage,0.7732696533203125},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,24324960},
   {mbcs_carriers_size,31457280}]},
 {{binary_alloc,2},
  [{sbcs_usage,1.0},
   {mbcs_usage,0.938345729714573},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,149064912},
   {mbcs_carriers_size,158859264}]},
 {{binary_alloc,0},
  [{sbcs_usage,1.0},
   {mbcs_usage,0.7732696533203125},
   {sbcs_block_size,0},
   {sbcs_carriers_size,0},
   {mbcs_block_size,24324960},
   {mbcs_carriers_size,31457280}]},

这是否意味着Erlang服务器有很多内存碎片？我该如何继续？

Answer 1

Erlang中的内存碎片比看上去复杂得多，这是一个已经解决的深层次问题。

看看弗雷德的这篇帖子https://blog.heroku.com/logplex-down-the-rabbit-hole#how-erlang-39-s内存-uff-works

它显示了整个问题，Erlang内存碎片没有得到解决，但可以减少。