我的HelloWorld队列是否正常工作? [英] Is my HelloWorld queue working?

问题描述

我将在应用程序中使用这种设计，但是我对python中的线程和队列方面的知识还很陌生.显然，实际的应用程序不是要打招呼，而是设计是相同的-也就是说，有一个过程需要花费一些时间来设置和拆除，但我一次就能完成多个任务.任务将在随机时间到达，并且通常会突然出现.

I'm about to put this design into use in an application, but I'm fairly new to threading and Queue stuff in python. Obviously the actual application is not for saying hello, but the design is the same - i.e. there is a process which takes some time to set-up and tear down, but I can do multiple tasks in one hit. Tasks will arrive at random times, and often in bursts.

这是明智且线程安全的设计吗?

Is this a sensible and thread safe design?

class HelloThing(object):

  def __init__(self):
    self.queue = self._create_worker()

  def _create_worker(self):
    import threading, Queue

    def worker():
      while True:
        things = [q.get()]
        while True:
          try:
            things.append(q.get_nowait())
          except Queue.Empty:
            break
        self._say_hello(things)
        [q.task_done() for task in xrange(len(things))]

    q = Queue.Queue()
    n_worker_threads = 1
    for i in xrange(n_worker_threads):
      t = threading.Thread(target=worker)
      t.daemon = True
      t.start()

    return q

  def _say_hello(self, greeting_list):
    import time, sys
    # setup stuff
    time.sleep(1)
    # do some things
    sys.stdout.write('hello {0}!\n'.format(', '.join(greeting_list)))
    # tear down stuff
    time.sleep(1)


if __name__ == '__main__':
  print 'enter __main__'

  import time
  hello = HelloThing()

  hello.queue.put('world')
  hello.queue.put('cruel world')
  hello.queue.put('stack overflow')

  time.sleep(2)

  hello.queue.put('a')
  hello.queue.put('b')

  time.sleep(2)

  for i in xrange(20):
    hello.queue.put(str(i))

  #hello.queue.join()

  print 'finish __main__'

推荐答案

1. 线程安全由Queue实现处理(如果需要，您也必须在_say_hello实现中进行处理).

1. The thread safety is handled by Queue implementation (also you must handle in your _say_hello implementation if it is required).

突发处理程序问题:突发仅应由单个线程处理.(例如:假设您的进程设置/拆卸需要10秒；从第1秒起，所有线程将在第1秒开始忙于突发.第二个5个新任务(或突发任务)，但是没有可用的线程来处理它们.因此，应通过特定时间窗的最大任务数(或无限")来定义突发.队列中的条目应该是任务列表.

Burst handler problem: A burst should be handled by a single thread only.(ex: let's say your process setup/teardown takes 10 seconds; at second 1 all threads will be busy with burst from sec 0, on second 5 a new task(or burst) but no thread available to handle them/it). So a burst should be defined by max number of tasks (or maybe "infinite") for a specific time-window. An entry in queue should be a list of tasks.

如何对突发任务列表进行分组? 我提供了一个解决方案作为代码，更容易解释...

How can you group burst tasks list? I provide a solution as code, more easy to explain ...

producer_q = Queue()
def _burst_thread():
   while True:
      available_tasks = [producer_q.get()]
      time.sleep(BURST_TIME_WINDOW)
      available_tasks.extend(producer_q.get() # I'm the single consumer, so will be at least qsize elements  
                             for i in range(producer_q.qsize()))
      consumer_q.push(available_tasks)

如果您希望突发中有最多的消息，则只需将available_tasks切片为多个列表.

If you want to have a maximum of messages in a burst, you just need to slice the available_tasks in multiple lists.

这篇关于我的HelloWorld队列是否正常工作?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！