带有SyncManager类的python中的多进程优先级队列的队列功能 [英] queue function for Multiprocess Priority Queue in python with SyncManager class

问题描述

我想实现多处理优先级队列. 我找到了这个答案:- Strange Queue.PriorityQueue Python 2.7.6中多处理的行为

I wanted to implement multiprocessing priorityqueue . I found this answer :- Strange Queue.PriorityQueue behaviour with multiprocessing in Python 2.7.6

通过 Dano

我实现了之后.我可以将.get()和.put()函数用于我的优先级队列，但是当我使用.queue打印队列中的当前元素时，它给了我一个错误

After I implemented this . I could use .get() and .put() function for my Priority Queue but when i used .queue to print the current elements in the queue it gave me an error

代码:-

 class MyManager(SyncManager):   
    pass

def get_manager():
    MyManager.register("PriorityQueue", PriorityQueue)  # Register a shared PriorityQueue
    m = MyManager()
    m.start()
    return m

m = get_manager()
call= m.PriorityQueue()
for i in range(5):
    call.put(i)

print(call.queue)

错误:AttributeError: 'AutoProxy[PriorityQueue]' object has no attribute 'queue'

我阅读了SyncManager的python文档并修改了我的代码.

I read the python documentation for SyncManager and modified my code .

新代码:-

class MyManager(SyncManager):   
    pass

def get_manager():
    MyManager.register("PriorityQueue", PriorityQueue,exposed=['put','get','queue'])  # Register a shared PriorityQueue
    m = MyManager()
    m.start()
    return m

m = get_manager()
call= m.PriorityQueue()
for i in range(5):
    call.put(i)

print(call.queue)

现在的输出是:-

<bound method AutoProxy[PriorityQueue].queue of <AutoProxy[PriorityQueue] object, typeid 'PriorityQueue' at 0x7ff3b48f2dd0>>

我仍然没有获得队列中的元素，我了解了寄存器函数的method_to_typeid属性，以映射exposed中提到的函数的返回类型，但是我不知道该如何使用.

I am still not getting the elements in the queue , i read about method_to_typeid attribute of register function to map the return type of functions mentioned in exposed , but i don't know how use that .

有人可以帮我吗，这样我就可以打印队列中的元素而不必将它们从队列中弹出

Can someone help me with this , so that i could print elements of the queue without poping them from queue

推荐答案

您只能通过代理使用引用对象的方法.由于PriorityQueue().queue不是方法，而是实例属性，因此您需要提供一种可以返回该属性值的方法. 下面的示例选择了带有PriorityQueue子类的通用get_attribute方法.

You can only use methods of a referent through a proxy. Since PriorityQueue().queue is not a method, but an instance attribute, you need to provide a method which can return the value of this attribute. The example below opts for a generalized get_attribute method with subclassing PriorityQueue.

# Python 3.7.1
from queue import PriorityQueue
from multiprocessing.managers import SyncManager
from multiprocessing import Process


SENTINEL = None


class MyPriorityQueue(PriorityQueue):
    def get_attribute(self, name):
        return getattr(self, name)


class MyManager(SyncManager):
    pass


def get_manager():
    MyManager.register("PriorityQueue", MyPriorityQueue)
    m = MyManager()
    m.start()
    return m


def f(q):
    for item in iter(lambda: q.get()[1], SENTINEL):
        print(item)
    print(f'queue: {q.get_attribute("queue")}')


if __name__ == '__main__':

    m = get_manager()
    pq = m.PriorityQueue()

    tasks = enumerate([f'item_{i}' for i in range(5)] + [SENTINEL])

    for task in tasks:
        pq.put(task)

    print(f'queue: {pq.get_attribute("queue")}')
    print(f'maxsize: {pq.get_attribute("maxsize")}')

    p = Process(target=f, args=(pq,))
    p.start()
    p.join()

示例输出:

queue: [(0, 'item_0'), (1, 'item_1'), (2, 'item_2'), (3, 'item_3'), (4, 'item_4'), (5, None)]
maxsize: 0
item_0
item_1
item_2
item_3
item_4
queue: []

这篇关于带有SyncManager类的python中的多进程优先级队列的队列功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！