Python多处理的简单方法来实现一个简单的计数器? [英] Python multiprocessing easy way to implement a simple counter?

问题描述

大家好，我现在在python中使用多重处理.我只是想知道是否存在某种简单的计数器变量，当每个进程完成某些任务后，它们会只是增加(有点像总共完成了多少工作).

Hey everyone, I am using multiprocessing in python now. and I am just wondering whether there exists some sort of simple counter variable that each process when they are done processing some task could just increment ( kind of like how much work done in total).

我查找了Value API，不要认为它是可变的.

I looked up the API for Value, don't think it's mutable.

推荐答案

Value确实是可变的；您可以从ctypes模块中指定所需的数据类型，然后可以对其进行突变.这是一个完整的可运行脚本，用于演示此内容:

Value is indeed mutable; you specify the datatype you want from the ctypes module and then it can be mutated. Here's a complete, working script that demonstrates this:

from time import sleep
from ctypes import c_int
from multiprocessing import Value, Lock, Process

counter = Value(c_int)  # defaults to 0
counter_lock = Lock()
def increment():
    with counter_lock:
        counter.value += 1

def do_something():
    print("I'm a separate process!")
    increment()

Process(target=do_something).start()
sleep(1)
print counter.value   # prints 1, because Value is shared and mutable

Luper在下面的注释中正确指出，默认情况下Value值是锁定的.从某种意义上说，这是正确的，即使一个分配包含多个操作(例如分配一个可能包含多个字符的字符串)，该分配也是原子的.但是，当增加一个计数器时，您仍然需要如我的示例中所提供的外部锁，因为递增会加载当前值，然后递增当前值，然后将结果分配回Value.

Luper correctly points out in a comment below that Value values are locked by default. This is correct in the sense that even if an assignment consists of multiple operations (such as assigning a string which might be many characters) then this assignment is atomic. However, when incrementing a counter you'll still need an external lock as provided in my example, because incrementing loads the current value and then increments it and then assigns the result back to the Value.

因此，没有外部锁定，您可能会遇到以下情况:

So without an external lock, you might run into the following circumstance:

• 进程1(以原子方式)读取计数器的当前值，然后对其进行递增
• 在进程1可以将递增的计数器分配回Value之前，将发生上下文切换
• 进程2(从原子上)读取计数器的当前(未递增)值，将其递增，然后将(递增)结果(原子上)分配回Value
• 第1步(原子地)分配其增量值，吹走了第2步执行的增量

• Process 1 reads (atomically) the current value of the counter, then increments it
• before Process 1 can assign the incremented counter back to the Value, a context switch occurrs
• Process 2 reads (atomically) the current (unincremented) value of the counter, increments it, and assigns the incremented result (atomically) back to Value
• Process 1 assigns its incremented value (atomically), blowing away the increment performed by Process 2

这篇关于Python多处理的简单方法来实现一个简单的计数器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！