垂直和水平平行 [英] Vertical and Horizontal Parallelism

问题描述

最近在并行领域工作，我知道有两个术语垂直并行性"和水平并行性".有人说openmp(共享内存并行性)为垂直，而mpi(分布式内存并行性)为水平并行.为什么将这些术语如此称呼?我不知道原因.这样称呼它们只是术语吗?

Recently working in parallel domain i come to know that there are two terms "vertical parallelism " and "horizontal parallelism". Some people says openmp ( shared memory parallelism ) as vertical while mpi ( distributed memory parallelism ) as horizontal parallelism. Why these terms are called so ? I am not getting the reason. Is it just terminology to call them so ?

推荐答案

这些术语似乎没有得到广泛使用，可能是因为经常某个进程或系统在不加区分的情况下同时使用这两个术语.这些概念非常通用，涵盖的范围远远超过MPI或openmp.

The terms don't seem to be widely used, perhaps because often time a process or system is using both without distinction. These concepts are very generic, covering much more than the realm of MPI or openmp.

垂直并行性是系统同时使用多个不同设备的能力.例如，一个程序可能有一个线程在做大量的计算，而另一个则在处理数据库查询，而第三个在做IO.大多数操作系统自然会暴露出这种能力.

Vertical parallelism is the faculty for a system to employ several different devices at the same time. For instance, a programme may have a thread doing heavy computation, while another is handling DB queries, and the third is doing IO. Most operating systems expose naturally this faculty.

当使用单个设备或对多个相似数据项执行操作时，就会发生水平并行性.例如当在同一段代码上运行多个线程但使用不同的数据时，就会发生这种并行性.

Horizontal parallelism occurs when a single device is used or operation is executed on several similar items of data. This is the sort of parallelism that happen for instance when running several threads on the same piece of code, but with different data.

在软件世界中，一个有趣的示例实际上是地图归约算法，该算法同时使用以下两种方法:

In the software world, an interesting example is actually the map reduce algorithm, which uses both:

• 水平并行发生在地图阶段，当数据分散并分散在多个cpu上进行处理时，

• horizontal parallelism occurs at the map stage, when data is split and scattered accross several cpu for processing,

垂直并行发生在map和reduce阶段之间，在此阶段，数据首先被分成块，然后由map线程进行处理，再由reduce线程进行累积，

vertical parallelism happens between the map and reduce stage, where data is first divided in chunks, then processed by the map threads, and accumulated by the reduce thread,

在硬件领域，超标量流水线CPU确实使用了两种变体，流水线是垂直并行化的一个特殊实例(就像map/reduce阶段一样，但还有更多步骤).

Similarily, in the hardware world, superscalar pipelined CPUs do use both variations, where pipelining is a particular instance of vertical parallelisation (just like the map/reduce staging, but with several more steps).

使用此术语的原因可能与供应链使用的原因相同:价值是通过链接不同的步骤或处理级别来产生的.最终产品可以看作是抽象结构树(从下至上)或依赖关系(从上至下)的根，其中每个节点都是中间级别或步骤的结果.您可以在这里轻松地看到供应链和计算之间的类比.

The reason behind the use of this terminology probably comes from the same reasons it is used with supply chains: values are produced by chaining different steps or levels of processing. The final product can be seen as the root of an abstract tree of constructions (from bottom to top) or dependency (from top to bottom) , where each node is the result of an intermediate level or step. You can easily see the analogy between supply chains and computation here.

这篇关于垂直和水平平行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！