Python队列即使在无限时也提高了满度 [英] Python Queue raising Full even when infinite

问题描述

嗯，我是一名Java开发人员，决心使用python 2.7创建应用程序

Well, I am a Java developer who resolved to create an application using python 2.7

基本上，我正在使用一个队列从UDP端口接收信息，另一个进程负责提取这些先前插入此队列的值.到目前为止，唯一的问题是文档( https://docs.python .org/2/library/queue.html )表示，针对无限队列时，基本上，我们在实例化Queue时应发送参数< = 0(例如Queue.Queue(maxsize = 0))，但是，当我的Queue包含大约32766个值时，它将引发/引发Full异常.

Basically I am using a Queue to receive information from a UDP port and another process is responsible to pull these values previously inserted out this Queue. So far so good, the only problem is, the documentation (https://docs.python.org/2/library/queue.html) says that when an infinite Queue is aimed, basically we should send a parameter <= 0 when instantiating the Queue (e.g Queue.Queue(maxsize=0)), however when my Queue comprises around 32766 values, it throws/raises a Full exception..

有人可以告诉我为什么吗?

Could anyone tell me why, please?

    while True:
        try:
            self.queue.put(valueToInsertIntoQueue, block=False)
            break
        except Full:
            print('WHYYY? Queue is supposed to be infinite, at least until my memory blows...')
            continue

TraceBack:

TraceBack:

Traceback (most recent call last):
  File "MyFile.py", line 71, in inserValueIntoQueue
    self.queue.put(valueToInsertIntoQueue, block=False)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 155, in put
    return self.put(obj, False)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 102, in put
    raise Full
Full

平台:

导入平台

import platform

platform.architecture()

platform.architecture()

('64bit'，'')

('64bit', '')

推荐答案

嗯，它在2 ^ 15-1左右失败的事实使我相信队列中的基础变量有问题.经过一些研究后，看来可能可能是管道出了点问题，尽管我不确定100％.以下几个链接可能会有用:

Hmm, the fact that it's failing at around 2^15 - 1 leads me to believe that it's something wrong with an underlying variable within the queue. After doing a little research, it seems like it might be something wrong with the pipe, although I'm not 100% sure. Here's a couple of links you might find useful:

http://bugs.python.org/issue8426

http://bugs.python.org/issue8237

很抱歉没有完全回答-我会发表评论，但我的声誉不够

Sorry for not answering fully -- I would have commented, but I don't have enough reputation

我看了源代码多处理队列.

Python中多处理队列中的值存储在Python Deque数据结构中.看看双端队列的底层C源代码 ，我注意到了双端队列的这种类型定义:

The values in a multiprocessing queue in Python are stored in a Python Deque data structure. Taking a look at the underlying C source code for the deque, I noticed this type definition for the deque:

typedef struct {
    PyObject_VAR_HEAD
    block *leftblock;
    block *rightblock;
    Py_ssize_t leftindex;       /* 0 <= leftindex < BLOCKLEN */
    Py_ssize_t rightindex;      /* 0 <= rightindex < BLOCKLEN */
    size_t state;               /* incremented whenever the indices move */
    Py_ssize_t maxlen;
    PyObject *weakreflist;
} dequeobject;

具体来说，请注意以下这一行:

Specifically, notice this line:

Py_ssize_t maxlen;

根据此PEP ，

引入了新的Py_ssize_t类型，其大小与编译器的size_t类型相同，但已签名.

A new type Py_ssize_t is introduced, which has the same size as the compiler's size_t type, but is signed.

因此，我的猜测是您的编译器的size_t类型为32位.有符号的32位整数的值范围在-32,768到32,767之间，上限是您的程序引发异常的位置.

So, my guess would be that your compiler's size_t type is 32 bits. A signed 32 bit integer has a range of values between -32,768 to 32,767, the upper bound being where your program is throwing an exception.

您是否有机会使用32位Python? :)

Are you by any chance using 32-bit Python? :)

这篇关于Python队列即使在无限时也提高了满度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！