你如何设计一个基于OTP-二郎/分布式容错多核系统的架构？ [英] How do you design the architecture of an Erlang/OTP-based distributed fault-tolerant multicore system?

问题描述

我想建立一个基于OTP-二郎/系统，解决了一个'embarassingly具有平行的问题。

I would like to build an Erlang/OTP-based system which solves an 'embarassingly parrallel' problem.

我已经读/脱脂通过：

• 了解您的一些二郎神;


• 编程二郎（阿姆斯特朗）;


• Erlang编程（利尼）;


• 二郎山/ OTP在行动。

我有过程的要点，消息，监事，gen_servers，日志记录等。

I have got the gist of Processes, Messaging, Supervisors, gen_servers, Logging, etc.

我明白，某些结构的选择取决于关注的应用，但我仍想知道的二郎山/ OTP系统设计的一些基本原则。

I do understand that certain architecture choices depend on the application in concern, but still I would like know some general principles of ERlang/OTP system design.

我是不是应该开始与上司几gen_servers，并逐步构造上？

Should I just start with a few gen_servers with a supervisor and incrementally build on that?

多少监事我应该有？如何确定系统的哪个部分应基于过程？我应该如何避免瓶颈？

How many supervisors should I have? How do I decide which parts of the system should be process-based? How should I avoid bottlenecks?

我应该加以后登录？

什么是二郎山/ OTP分布式容错多处理器系统结构的一般方法？

推荐答案

您错过的一个关键组成部分在这里二郎架构：应用程序！ （即，OTP应用程序，而不是软件应用程序的概念）。

Should I just start with a few gen_servers with a supervisor and incrementally build on that?

You're missing one key component in Erlang architectures here: applications! (That is, the concept of OTP applications, not software applications).

想想应用作为组件。在你的系统中的组件解决特定的问题，负责一套连贯的资源或抽象的东西从系统中重要或复杂的。

Think of applications as components. A component in your system solves a particular problem, is responsible for a coherent set of resources or abstract something important or complex from the system.

设计一个Erlang系统时的第一个步骤是决定需要哪些应用程序。有些可以从网站，因为它们是被拉到，这些大家可以参考图书馆。你需要自己写别人（否则你就不需要这种特殊的系统）。我们通常所说的业务逻辑这些应用程序（经常需要写一些图书馆自己为好，但要保持库以及配合一切融合在一起的核心业务应用之间的区别是非常有用）。

The first step when designing an Erlang system is to decide which applications are needed. Some can be pulled from the web as they are, these we can refer to as libraries. Others you'll need to write yourself (otherwise you wouldn't need this particular system). These applications we usually refer to as the business logic (often you need to write some libraries yourself as well, but it is useful to keep the distinction between the libraries and the core business applications that tie everything together).

您应该对每种过程要监视一名监事。

You should have one supervisor for each kind of process you want to monitor.

一堆相同的临时工？一名监事统治他们。

A bunch of identical temporary workers? One supervisor to rule them all.

不同的工艺与不同的责任和重启策略？一种每个不同类型的过程中，在正确的层次结构主管（取决于当事情应该重新启动，什么其他的进程需要下井带他们？上）。

Different process with different responsibilities and restart strategies? A supervisor for each different type of process, in a correct hierarchy (depending on when things should restart and what other process needs to go down with them?).

有时，没关系放一堆不同的工艺类型相同的超级之下。这在通常情况下，当你有几个单身的进程（例如，一个HTTP服务器的主管，一是ETS表的所有者的过程，一是统计收集器），将始终运行。在这种情况下，它可能是太大克鲁夫特具有用于每一个主管，因此它是通用的添加所述一个监下。要知道这样做时，使用特定的重启策略的影响，所以你不记下你的统计方法，例如，如果你的Web服务器崩溃（ one_for_one 在这样的情况下），以使用最常见的策略。

Sometimes it is okay to put a bunch of different process types under the same supervisor. This is usually the case when you have a few singleton processes (e.g. one HTTP server supervisor, one ETS table owner process, one statistics collector) that will always run. In that case, it might be too much cruft to have one supervisor for each, so it is common to add the under one supervisor. Just be aware of the implications of using a particular restart strategy when doing this, so you don't take down your statistics process for example, in case your web server crashes (one_for_one is the most common strategy to use in cases like this).

在你的系统中每个并发活动应该在它自己的过程。有并发的错误抽象被二郎神系统设计之初最常见的错误。

Every concurrent activity in your system should be in it's own process. Having the wrong abstraction of concurrency is the most common mistake by Erlang system designers in the beginning.

很多人并不能用来处理并发;他们的系统往往太少了。一过程中，或数个巨大的人，在运行中的序列的一切。这些系统通常是满code嗅觉和code是非常严格的，难以重构。这也使得他们更慢，因为他们可能无法使用所有可用的核心二郎神。

Some people are not used to deal with concurrency; their systems tend to have too little of it. One process, or a few gigantic ones, that runs everything in sequence. These systems are usually full of code smell and the code is very rigid and hard to refactor. It also makes them slower, because they may not use all the cores available to Erlang.

其他人即时掌握并发概念，但未能以最佳方式应用它们;他们的系统倾向于过度使用过程中的概念，使得许多过程赋闲等待别人正在做的工作。这些系统往往是不必要的复杂和难以调试

Other people immediately grasp the concurrency concepts but fail to apply them optimally; their systems tend to overuse the process concept, making many process stay idle waiting for others that are doing work. These systems tend to be unnecessarily complex and hard to debug.

在本质上，这两个变种你得到了同样的问题，你不使用所有可用的并发给你，你没有得到最大的表现出来的制度。

In essence, in both variants you get the same problem, you don't use all the concurrency available to you and you don't get the maximum performance out of the system.

如果你坚持href=\"http://www.codinghorror.com/blog/2007/03/curlys-law-do-one-thing.html\" rel=\"nofollow\">单一职责原则以及由规则恪守对每一个过程的真正的系统中的并发活动，你应该没问题。

If you stick to the single responsibility principle and abide by the rule to have a process for every truly concurrent activity in your system, you should be okay.

很难说，取决于你的系统和它做什么的非常多。但是总的来说，如果你有应用程序之间的责任划分好，你应该能够扩展，似乎从系统的其余部分分别是瓶颈中的应用。

Hard to say, depends very much on your system and what it's doing. Generally though, if you have a good division of responsibility between applications you should be able to scale the application that appears to be the bottleneck separately from the rest of the system.

这里的金科玉律是的测量，测量，测量的！不要以为你有什么要改进，直到你已经测得的。

The golden rule here is to measure, measure, measure! Don't think you have something to improve until you've measured.

Erlang是因为它可以让你躲在接口并发（被称为隐并发）大。例如，您可以使用一个功能模块的API，一个正常模块：函数（参数）接口，这可能反过来催生成千上万的过程，而不必知道调用者。如果你有你的抽象和你的API的权利，你总是可以并行化或优化库你已经开始使用它了。

Erlang is great in that it allows you to hide concurrency behind interfaces (known as implicit concurrency). For example, you use a functional module API, a normal module:function(Arguments) interface, that could in turn spawn thousands of processes without the caller having to know that. If you got your abstractions and your API right, you can always parallelize or optimize a library after you've started using it.

话虽这么说，这里有一些一般性的指导方针：

That being said, here are some general guide lines:

• 尝试直接发送邮件给收件人，避免窜或通过中介进程的路由信息​​。否则系统只是花时间到处移动信息（数据），而没有真正的工作。


• 请不要过度使用OTP设计模式，如gen_servers。在很多情况下，你只需要启动一个进程，运行一些片code，然后退出。对于这一点，一个gen_server是矫枉过正。

和一份奖励的建议是：不要重复使用过程。产卵在二郎一个过程是如此的廉价和快速的，它没有任何意义，一旦其生命周期结束再利用的过程。在某些情况下，它可能是有意义再次使用的状态（一档例如复杂的分析），但更好标准地存储在其他地方（在ETS表，数据库等）。

And one bonus advice: don't reuse processes. Spawning a process in Erlang is so cheap and quick that it doesn't make sense to re-use a process once its lifetime is over. In some cases it might make sense to re-use state (e.g. complex parsing of a file) but that is better canonically stored somewhere else (in an ETS table, database etc.).

有在二郎山/ OTP一些基本的记录功能不已，错误记录器。你可以加上 SASL （系统架构支持库）起床，在记录没有运行 - 时间。

There's some basic logging functionality in Erlang/OTP already, the error logger. Together with SASL (System Architecture Support Libraries) you can get up and running with logging in no-time.

到时候（如果你从一开始就抽象日志API），你可以的东西，更好地满足您的需要交流这一点。今天的事实上的第三方日志库是芭蕉拉格。

When the time comes (and if you've abstracted the logging API from the beginning) you could exchange this for something that better fits your needs. The de-facto 3rd party logging library today is Basho's Lager.

要总结一下一直在上面说：

To summarize what's been said above:

• 把你的系统到应用程序


• 放入正确的监管层次的过程，这取决于他们的需求和依赖性


• 对每一个真正的并发活动的过程中系统


• 保持朝向在系统中的其他部件的功能的API。这使您可以：
  
  
  • 重构您的code，而不改变目前正使用它的code
  
  
  • 优化code事后
  
  
  • 当需要时分发系统（只是做给后面的API另一个节点的来电！来电者不会注意到！）
  
  
  • 更方便地测试code（较少的工作设置测试工具，更容易理解如何使用它）
  
  

  常见的问题：

  
  
  • 进程太多
  
  
  • 太少流程
  
  
  • 太多的路由（转发的消息，链接的过程）
  
  
  • 太少的应用程序（我从来没有见过相反的情况下​​，实际上）
  
  
  • 没有足够的抽象（使人很难重构和推理。这也使得它很难对其进行测试！）
  
  

  这篇关于你如何设计一个基于OTP-二郎/分布式容错多核系统的架构？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！