在Erlang比单线程Java更快之前需要多少CPU [英] How many CPUs are needed before Erlang is faster than single-threaded Java

问题描述

我正在使用Java，我已经在网上阅读了很多关于Erlang的内容，我有两个大问题：

1. 那么Erlang会比Java简单多少（如果有的话）？
   
   我假设这个Java将会从射击基准（Erlang不这样做）。那么，我需要多少个CPU来使Erlang在单线程Java上发光（在下面给出的特定情况下）？




2. 阅读后关于Erlang一段时间，我遇到了一些评论/帖子，说最大的Erlang系统包含大量的C / C ++。
   
   这是为了速度的原因（我的假设）或别的什么为什么要这样？

我已经阅读了大多数机器上的处理器数量，线程模型是很难（我同意），但我正在寻找什么时候，线将被越过，以便我可以在正确的时间改变语言/范式。

有一点背景/背景：

我在Java服务器上工作，这是非常CPU限制和容易并行的。这通常是通过单个传入更新（通过TCP）触发对多个（100s）输出的更改。

计算通常很简单（几个循环，只需要很多的算术运算），输入即可快速（100 / s）。

目前我们正在运行4台CPU机器，并在每个机器上运行多个服务所以多线程是相当毫无意义的，而Java似乎运行更快，没有同步块等需要使其多线程）。现在有一个强大的速度推动，我们现在可以访问24个处理器机器（如果需要，每个进程），所以我想知道如何最好地进行 - 大规模多线程Java或更容易编码的东西，如Erlang。

解决方案

由于这是一个算术繁重的工作负载，您已经完成了将代码分解成单独的服务流程的工作，您将无法获得很多来自Erlang。你的工作似乎很适合Java。 Erlang善于微小的交易 - 例如msg切换或提供静态或简单动态网页。不是在企业号码处理或数据库工作负载。但是，您可以使用外部数字库和数据库，并使用Erlang作为MSG交换机：D这是couch-db的作用：P

- 编辑 -

1. 如果将算术运算移动到Erlang异步IO司机erlang将会像语言拍摄的东西一样好 - 但是24个cpu也许并不重要， erlang数据库是程序性的，而且相当快 - 可以在您的应用程序中利用每个事务更新100个实体。




2. erlang运行时系统需要C和C ++的混合，因为（a）erlang仿真器是用C / C ++编写的（你必须从某个地方开始），（b）你必须和内核谈谈做异步文件io和网络io，和（c）系统的某些部分需要快速起泡 - 数据库系统的后台（健忘）。

- 讨论 -

与使用共享内存总线的6核* 4 CPU拓扑中的24个CPU - 您有4个NUMA实体（CPU）和一个中央记忆你需要对范式有所了解，无共享的多进程方法可能会杀死你的内存总线。为了解决这个问题，您需要创建具有6个处理线程的4个进程，并将每个处理线程绑定到相应CPU中的相应核心。这6个线程需要做协作的多线程 - Erlang和 Lua 有这个天生的 - Erlang它是一个硬核的方式，因为它有一个完整的调度程序作为其运行时的一部分，它可以用来创建任意数量的进程。现在，如果要在4个进程中分配任务（每个物理CPU 1个），那么您将是一个快乐的人，但是您正在运行4个Java VM（可能是）认真的工作（yuck，由于很多原因）。这个问题需要用更好的切片和骰子的能力来解决。

在Erlang OTP系统中，它是专为冗余的强大的网络系统而设计的，但现在它正在向同一机型的NUMA-esque CPU移动。它已经有一个踢脚踢SMP模拟器，它将很快成为NUMA感知。通过这种编程模式，您可以有更好的机会使您的强大的服务器饱和，而不会导致您的公共汽车死亡。

也许这个讨论已经是理论性的;然而，当您获得8x8或16x8拓扑结构时，您也将为此准备好。所以我的答案是当你的主板上有两个现代的物理CPU时，你应该考虑一个更好的编程范例。

作为以下讨论的主要产品的示例：微软的SQL Server是在数据库引擎构建的SQL-OS层中的CPU级NUMA。

I am currently using Java, I've read a lot about Erlang on the net, and I have 2 big questions:

1. How much slower (if any) will Erlang be over simple Java?
   I'm assuming here that Java is going to be faster from the shootout benchmarks on the net (Erlang doesn't do that well). So, how many more CPUs am I going to need to make Erlang shine over single-threaded Java (in my particular situation, given below)?

2. After reading around about Erlang for a while I've hit on a number of comments/posts that say that most large Erlang systems contain a good amount of C/C++.
   Is this for speed reasons (my assumption) or something else? i.e. Why is this required?

I have read about the number of processors in most machines going up and threading models being hard (I agree) but I am looking to find out when the "line" is going to be crossed so that I can change language/paradigm at the right time.

A bit of background/context:
I am working server-side on Java services which are very CPU-bound and easily made parallel. This is due to, typically, a single incoming update (via TCP) triggering a change to multiple (100s of) outputs.

The calculations are typically pretty simple (few loops, just lots of arithmetic) and the inputs are coming in pretty fast (100/s).

Currently we are running on 4 CPU machines and running multiple services on each (so multi-threading is pretty pointless and Java seems to run faster without the sync blocks, etc required to make it multi-threaded). There is now a strong push for speed and we now have access to 24 processor machines (per process if required) so I am wondering how best to proceed - massively multi-threaded Java or something easier to code, like Erlang.

解决方案

since this is a arithmetic heavy workload and you have already done the job of splitting out the code into seperate service processes, you wouldn't gain much from Erlang. Your job seems to fit Java comfortably. Erlang is good at tiny transactions -- such as msg switching or serving static or simple-dynamic web-pages. Not -- inately at enterprise number-crunching or database workload.

However, you could build on external numerical libraries and databases and use Erlang as a MSG switch :D that's what couch-db does :P

-- edit --

1. If you move your arithmetic operations into an Erlang async-IO driver erlang will be just as good as the language shoot-out stuff -- but with 24 cpu's perhaps it won't matter that much; the erlang database is procedural and thefore quite fast -- this can be exploited in your application updating 100 entities on each transaction.

2. The erlang runtime system needs to be a mix of C and C++ because (a) the erlang emulator is written in C/C++ (you have to start somewhere), (b) you have to talk to the kernel to do async file io and network io, and (c) certain parts of the system need to be blistering fast --e.g., the backend of the database system (amnesia).

-- discussion --

with 24 CPU's in a 6 core * 4 CPU topology using a shared memory buss -- you have 4 NUMA entities (the CPUs) and one central memory. You need to be wise about the paradigm, the shared-nothing multi-process approach might kill your memory buss.

To get around this you need to create 4 processes with 6 processing threads and bind each processing thread the corresponding core in the corresponding CPU. These 6 threads need to do collaborative multi-threading -- Erlang and Lua have this innately -- Erlang does it in a hard-core way as it has a full-blown scheduler as part of its runtime which it can use to create as many processes as you want.

Now if you were to partition your tasks across the 4 processes (1 per physical CPU) you would be a happy man, however you are running 4 Java VM's doing (presumably) serious work (yuck, for many reasons). The problem needs to be solved with a better ability to slice and dice the problem.

In comes the Erlang OTP system, it was designed for redundant robust networked systems, but now it is moving towards same-machine NUMA-esque CPU's. It already has a kick-ass SMP emulator, and it will become NUMA aware as well soon. With this paradigm of programming you have a much better chance to saturate your powerful servers without killing your bus.

Perhaps this discussion has been theoretical; however, when you get a 8x8 or 16x8 topology you will be ready for it as well. So my answer is when you have more then 2 -- modern -- physical CPU's on your mainboard you should probably consider a better programming paradigm.

As an example of a major product following the discussion here: Microsoft's SQL Server is CPU-Level NUMA-aware in the SQL-OS layer on which the database engine is built.

这篇关于在Erlang比单线程Java更快之前需要多少CPU的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！