二郎：分布式的阵列上工作 [英] Erlang: Distributed work on an array

问题描述

我工作的一个项目，我们有原子排列充当哈希值。每当用户连接到某个值被散列的服务器，该散列被用作索引来查找阵列中的元件，并返回该元素。 外部势力（这是由一个长期运行的gen_server处理）是能够改变这阵，所以我不能简单地硬code吧。我的问题是如何举办这阵。

I'm working on a project where we have an array of atoms which acts as a hash. Whenever a user connects to the server a certain value is hashed, and that hash is used as an index to lookup the element in the array, and return that element. "Outside forces" (which are handled by a long-running gen_server) are able to change this array, so I can't simply hardcode it. My problem is how to "host" this array.

我的第一个执行是一个简单的gen_server从而保持围绕阵列的副本，并将其发送给谁要求它。然后要求它的过程可能穿越它，并得到他们想要的索引。这种实现方式具有和内存过多的被使用，这是我归因于若本同一阵列漂浮的许多副本。

My first implementation was a simple gen_server which kept a copy of the array around and sent it to whoever asked for it. The process asking for it could then traverse it and get the index they want. This implementation had and inordinate amount of memory being used, which I attributed to there being so many copies of this same array floating around.

我目前的实现有一个中央gen_server负责处理此阵的状态，并处理实际要求孩子。当状态改变中央gen_server更新的孩子们。当一个进程想要找到它的散列结果它发送它的索引编号以中央gen_server，其将请求转发给孩子之一。子穿过它的本地列表中，并且将所得的原子回原始过程

My current implementation has a central gen_server which handles the state of this array, and children which handle the actual requests. When the state changes the central gen_server updates the children. When a process wants to find it's hash result it sends its index number to the central gen_server, which forwards the request to one of the children. The child traverses its "local" list, and sends the resulting atom back to the original process.

与当前执行的问题是，它能够在高流量越陷越深。我已经使用越来越多的孩子尝试过，但我pretty确保中央gen_server是瓶颈。

The problem with the current implementation is that it gets bogged down at high traffic. I've tried using more and more children, but I'm pretty sure the central gen_server is the bottleneck.

有没有人有一个更好的解决方案，任何想法我的问题？

Does anyone have any ideas on a better solution to my problem?

编辑：％s /阵列/列表/ G

%s/array/list/g

推荐答案

我建议你使用 ETS表。我认为阵列方法不太有效。随着 ETS表，因为应用程序后端在公众创建，任何进程都可以，只要它需要它查找的项目。 ETS表在二郎山目前新版本有并发访问的能力。

I suggest that you use ETS Tables.I think that the Array method is not efficient enough. With an ETS Table, created as public within the application backend, any process can lookup an item as soon as it needs it. ETS Tables in the current newer versions of erlang have the capability for concurrent access.

%% Lets create a record structure 
%% where by the key will be a value
%% in the array.
%% For now, i do not know what to 
%% put in the field: 'other'

-record(element,{key,other}).

create_table(TableName)->
    Options = [
                named_table,set,
                public,
                {keypos,2}, %% coz we are using record NOT tuple
                {write_concurrency,true}
            ],
    case ets:new(TableName,Options) of
        TableName -> {success,true};
        Error -> {error,Error}
    end.

lookup_by_hash(TableName,HashValue)->
    try ets:lookup(TableName,HashValue) of
        Value -> {value,Value};
    catch
        X:Y -> {error,{X,Y}}
    end.

有了这样的安排，你会避免单一故障点从单一gen_server保存数据所产生的。此数据由许多进程需要，因此，不应由一个单一过程进行。这就是通过在只要它需要做出一个查找任何时候任何进程访问的表。到网价值观在数组应转换到窗体的记录为元素，然后在 ETS表插入。搜索结果这种方法的搜索结果1的优势。我们可以创建许多 ETS表尽可能页2。一个ETS表可以处理许多比一个数据结构以上的元素，例如一个清单或具有低得多的可比存储器消耗的数组。结果，3。 ETS表可并发触手可及的任何进程访问，因此你不需要一个中央处理或服务器来处理数据
结果4。一个进程或gen_server保存这些数据，这意味着如果妥协（下降是由于一个完整的邮箱），这将是不可用的，因此这就需要阵列将不得不等待这一个服务器可以重新开始或过程我不知道....结果5。通过发送请求消息加上犯同样的阵列复制到需要它的每个进程访问数组数据不是Erlangic设计。结果6。最后， ETS表可以拥有从工艺处理转移。当拥有进程崩溃（仅gen_servers可以检测，他们正在死去[注意到这一点]），它可以在 ETS表转移到另一个进程接管。查看此处： ETS放弃搜索结果，这是我的想法。

With this kind of arrangement, you will avoid A Single Point of Failure arising from a single gen_server holding data. This data is needed by many processes and hence should not be held by a single process. That's where a Table accessible by any process at any time as soon as it needs to make a look up.

The Values in the Array should be converted to records of the form as element and then inserted in the ETS Tables.

Advantages of this approach

1. We can create as many ETS Tables as possible
2. An ETS Table can handle many more elements than a data structure such as a list or an Array with much lower comparable memory consumption.
3. ETS Tables can be concurrently accessed by any process within reach and hence you will not need a central process or server to handle data
4. A single process or gen_server holding this data, means that if its compromised (goes down due to a full mail box), it will be unavailable, hence the processes which need the array will have to wait for this one server to either restart or i dont know....
5. Accessing the Array data by sending request messages plus making copies of the same array to each process that needs it is not "Erlangic" design.
6. Finally, ETS Tables ownership can be transferred from process to process. When the owning process is crashing (Only gen_servers can detect that they are dying [take note of this]), it can transfer the ETS Table to another process to take over. Check here: ETS Give Away

That's my thinking.

这篇关于二郎：分布式的阵列上工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！