Jython中隐藏的多线程瓶颈？ [英] Hidden Multithreading Bottlenecks in Jython?

问题描述

在Jython中可以阻塞多线程/并行性的一些常见隐藏事物是什么？我有一些并行代码（使用Python的线程库），它不会扩展到超过3-4个CPU，我确信这不是因为任何这些明显的陷阱：

What are some common hidden things that can bottleneck multithreading/parallelism in Jython? I have some parallel code (using Python's threading library) that won't scale past 3-4 CPUs, and I'm sure it's not because of any of these obvious pitfalls:

• 显式锁

• Explicit locks

调用需要同步的库代码（我尝试并行化的算法基本上是写的从头开始并且不使用任何库。）

Calling library code that requires synchronization (The algorithm I'm trying to parallelize is basically written from scratch and doesn't use any libraries.)

基本上所有的算法都是一堆字符串处理，列表和字典查找和数学。我的理解是，与CPython不同，Jython没有GIL。

Basically all the algorithm does is a bunch of string processing, list and dictionary lookups and math. My understanding is that, unlike CPython, Jython does not have a GIL.

推荐答案

访问变量是那些隐藏的瓶颈之一。如果所有线程都访问某些共享数据结构，则线程之间将存在同步。

Accessing variables is one of those "hidden" bottlenecks. If all threads access some shared datastructure(s) there will be synchronization between the threads.

Jython努力实现与CPython的语言兼容性。 GIL确保的一件事是访问本地/全局变量，对象成员，dict元素（技术上本地，全局和对象成员也是dict元素）甚至列表元素都是原子的。为了避免用户出现意外，Jython使用并发哈希映射来实现dicts。这意味着在访问Jython中的任何类型的dict元素时会发生一些同步。这个sycnhronization是条带化的，支持从多个线程访问dict而不会阻塞它们，但是如果多个线程访问同一个变量，它们将会遇到同一个锁。

Jython tries hard to achieve language compatibility with CPython. One thing that the GIL ensures is that access to local/global variables, object members, dict elements (technically locals, globals and object members are also dict elements) or even list elements are atomic. To avoid surprises for users Jython uses a concurrent hash map to implement dicts. This means that there is some synchronization going on when accessing any kind of dict elements in Jython. This sycnhronization is striped to support access to the dict from multiple threads without blocking them, but if multiple threads access the same variable they are going to hit the same lock.

在Jython和任何其他语言中实现可伸缩性的最佳方法是确保您在每个线程中访问的数据也不会从其他线程访问。

The best way to achieve scalability in Jython, and any other language, is to make sure that the data you are accessing in each thread is not accessed from other threads as well.

这篇关于Jython中隐藏的多线程瓶颈？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！