Erlang：构建大型数据结构后，erl shell挂起 [英] Erlang: erl shell hangs after building a large data structure

问题描述

如以前的问题，我尝试使用Erlang proplist 来实现前缀trie。

代码似乎工作正常。 ..但是，由于某种原因，它不能很好地与交互式shell。当我尝试运行它时，shell挂起：

 
> Trie = trie：from_dict（）。 ％从字典中创建一个特技
％...打印的特技... 
％然后没有发生
 

我看到新的特里打印到屏幕上（即调用 trie：from_dict（）已经返回），然后shell挂起。没有新的> 提示出现， ^ g 不做任何事情（但 ^ c / code code code code code code code $ c $ / share / dict / words ），挂起只持续一两秒（这个特技几乎立即建立起来）...但是似乎随着字典的大小而成倍增长（100个字占5或10秒钟，我没有耐心尝试更大的单词列表）。另外，当shell挂起时，我注意到 beam.smp 进程开始消耗内存的一个批次 ）

那么有什么可以引起这个shell挂起和令人难以置信的内存使用吗？

---

一些各种意见：

• 我有一个希望，垃圾收集器在错误，但我不知道如何配置或创建一个实验来测试。




• 我尝试使用 eprof




• 这是我的添加字符串到trie功能：

 
 add（[]，Trie） - > 
 [stop | Trie]; 
 
 add（[Ch | Rest]，Trie） - > 
 SubTrie = proplists：get_value（Ch，Trie，[]），
 NewSubTrie = add（Rest，SubTrie） ，
 NewTrie = [{Ch，NewSubTrie} | Trie]，
％一旦获得
％超过60对，任意决定压缩键/值列表。 
如果长度（NewTrie）> 60  - > 
 proplists：compact（NewTrie）; 
 true  - > 
 NewTrie 
结束。 
 

解决方案

问题是（其中包括 - 看我的评论），你总是添加一个新的{Ch，NewSubTrie}元组到你的proplist Tries，无论Ch是否已经存在，否则。

而不是

  NewTrie = [{Ch，NewSubTrie} | 
  

你需要这样的东西：

<$ $ new $ {code code code code code code code $ c $

As suggested in answers to a previous question, I tried using Erlang proplists to implement a prefix trie.

The code seems to work decently well... But, for some reason, it doesn't play well with the interactive shell. When I try to run it, the shell hangs:

> Trie = trie:from_dict(). % Creates a trie from a dictionary
% ... the trie is printed ...
% Then nothing happens

I see the new trie printed to the screen (ie, the call to trie:from_dict() has returned), then the shell just hangs. No new > prompt comes up and ^g doesn't do anything (but ^c will eventually kill it off).

With a very small dictionary (the first 50 lines of /usr/share/dict/words), the hang only lasts a second or two (and the trie is built almost instantly)... But it seems to grow exponentially with the size of the dictionary (100 words takes 5 or 10 seconds, I haven't had the patience to try larger wordlists). Also, as the shell is hanging, I notice that the beam.smp process starts eating up a lot of memory (somewhere between 1 and 2 gigs).

So, is there anything obvious that could be causing this shell hang and incredible memory usage?

---

Some various comments:

• I have a hunch that the garbage collector is at fault, but I don't know how to profile or create an experiment to test that.

• I've tried profiling with eprof and nothing obvious showed up.

• Here is my "add string to trie" function:

add([], Trie) ->
    [ stop | Trie ];

add([Ch|Rest], Trie) ->
    SubTrie = proplists:get_value(Ch, Trie, []),
    NewSubTrie = add(Rest, SubTrie),
    NewTrie = [ { Ch, NewSubTrie } | Trie ],
    % Arbitrarily decide to compress key/value list once it gets
    % more than 60 pairs.
    if length(NewTrie) > 60 ->
            proplists:compact(NewTrie);
        true ->
            NewTrie
    end.

解决方案

The problem is (amongst others ? -- see my comment) that you are always adding a new {Ch, NewSubTrie} tuple to your proplist Tries, no matter if Ch already existed, or not.

Instead of

NewTrie = [ { Ch, NewSubTrie } | Trie ]

you need something like:

NewTrie = lists:keystore(Ch, 1, Trie, {Ch, NewSubTrie})

这篇关于Erlang：构建大型数据结构后，erl shell挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！