python GIL的多线程示例 [英] A multi-threading example of the python GIL

问题描述

我已经阅读了很多关于编写多线程代码时 Python GIL 业务有多糟糕"的文章，但我从未见过示例.有人可以给我一个基本的例子，说明在使用线程时 GIL 何时会导致问题.

I've read a quite a bit about how "bad" this python GIL business is when writing multi-threaded code, but I've never seen an example. Could someone please give me a basic example of when the GIL causes problems when using threading.

谢谢！

推荐答案

多线程的主要原因之一是程序可以利用多个 CPU(和/或 CPU 上的多个内核)来计算更多每秒操作数.但是在 Python 中，GIL 意味着即使您有多个线程同时进行计算，在任何给定时刻实际上只有一个线程在运行，因为所有其他线程都将被阻塞，等待获取全局解释器锁.这意味着 Python 程序的多线程版本实际上会比单线程版本慢，而不是更快，因为一次只运行一个线程——此外还有由强制每个线程每隔几毫秒等待、获取和放弃 GIL(循环方式).

One of the main reasons for multithreading is so that a program can take advantage of multiple CPUs (and/or multiple cores on a CPU) in order to compute more operations per second. But in Python, the GIL means that even if you have multiple threads working simultaneously on a computation, only one of those threads will actually be running at any given instant, because all of the other ones will be blocked, waiting to acquire the global interpreter lock. That means that a multithreaded version of a Python program will actually be slower than the single-threaded version, rather than faster, since only one thread runs at a time -- plus there is the accounting overhead incurred by forcing every thread to wait for, acquire, and then relinquish the GIL (round-robin style) every few milliseconds.

为了证明这一点，这里有一个玩具 Python 脚本，它产生指定数量的线程，然后作为它的计算"每个线程只是不断地增加一个计数器，直到 5 秒过去了.最后，主线程统计发生的计数器增量的总数并打印总数，以便我们衡量工作"有多少.在 5 秒内完成.

To demonstrate this, here is a toy Python script that spawns a specified number of threads, and then as its "computation" each thread just continually increments a counter until 5 seconds have passed. At the end, the main thread tallies up the total number of counter-increments that occurred and prints the total, to give us a measurement of how much "work" was done during the 5-second period.

import threading
import sys
import time

numSecondsToRun = 5

class CounterThread(threading.Thread):
   def __init__(self):
      threading.Thread.__init__(self)
      self._counter = 0
      self._endTime = time.time() + numSecondsToRun

   def run(self):
      # Simulate a computation on the CPU
      while(time.time() < self._endTime):
         self._counter += 1

if __name__ == "__main__":
   if len(sys.argv) < 2:
      print "Usage:  python counter 5"
      sys.exit(5)

   numThreads = int(sys.argv[1])
   print "Spawning %i counting threads for %i seconds..." % (numThreads, numSecondsToRun)

   threads = []
   for i in range(0,numThreads):
      t = CounterThread()
      t.start()
      threads.append(t)

   totalCounted = 0
   for t in threads:
      t.join()
      totalCounted += t._counter
   print "Total amount counted was %i" % totalCounted

.... 这是我在我的电脑上得到的结果(这是一台启用了超线程的双核 Mac Mini，FWIW):

.... and here are the results I get on my computer (which is a dual-core Mac Mini with hyper-threading enabled, FWIW):

$ python counter.py 1
Spawning 1 counting threads for 5 seconds...
Total amount counted was 14210740

$ python counter.py 2
Spawning 2 counting threads for 5 seconds...
Total amount counted was 10398956

$ python counter.py 3
Spawning 3 counting threads for 5 seconds...
Total amount counted was 10588091

$ python counter.py 4
Spawning 4 counting threads for 5 seconds...
Total amount counted was 11091197

$ python counter.py 5
Spawning 5 counting threads for 5 seconds...
Total amount counted was 11130036

$ python counter.py 6
Spawning 6 counting threads for 5 seconds...
Total amount counted was 10771654

$ python counter.py 7
Spawning 7 counting threads for 5 seconds...
Total amount counted was 10464226

注意第一次迭代是如何获得最佳性能的(其中只产生了一个工作线程)；当同时运行多个线程时，计数效率会显着下降.这显示了 Python 中的多线程性能如何被 GIL 削弱——用 C(或任何其他没有 GIL 的语言)编写的相同程序在运行更多线程时会显示出更好的性能，而不是更糟(直到工作线程的数量与当然，硬件上的内核数).

Note how the best performance was achieved by the first iteration (where only a single worker thread was spawned); the counting-productivity dropped considerably when more than one thread was running at once. This shows how multithreading performance in Python is crippled by GIL -- the same program written in C (or any other language without a GIL) would show much better performance with more threads running, not worse (up until the number of worker threads matched the number of cores on the hardware, of course).

这并不意味着多线程在 Python 中完全没有用——它在大多数或所有线程被阻塞等待 I/O 而不是 CPU 受限的情况下仍然有用.这是因为被阻塞等待 I/O 的 Python 线程在等待时不会保持 GIL 锁定，因此在此期间其他线程仍然可以自由执行.但是，如果您需要并行化计算密集型任务(例如，光线跟踪或计算 Pi 的所有数字或密码破解或类似的)，那么您将需要使用多个进程而不是多个线程，或者使用不同的语言没有 GIL.

This doesn't mean that multithreading is completely useless in Python, though -- it's still useful in cases where most or all of your threads are blocked waiting for I/O rather than CPU-bound. That's because a Python thread that is blocked waiting for I/O doesn't hold the GIL locked while it waits, so during that time other threads are still free to execute. If you need to parallelize a computation-intensive task, though (e.g. ray tracing or computing all the digits of Pi or codebreaking or similar), then you'll want to either use multiple processes rather than multiple threads, or use a different language that doesn't have a GIL.

这篇关于python GIL的多线程示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！