芹菜即使允许腌制也不接受腌制 [英] Celery not accepting pickle even after allowing it

问题描述

我正在尝试编写一个芹菜应用程序，它将Numpy数组(或任何任意对象)传递给Worker。据我所知，这需要通过pickle进行序列化(注意：我知道安全问题，但在本例中这不是问题)。

但是，即使在尝试了所有我能找到的允许Pickle作为序列化程序的方法之后，我仍然得到以下Kombu异常：

kombu.exceptions.ContentDisallowed: Refusing to deserialize untrusted
content of type pickle (application/x-python-serialize)

我的当前文件当前为：

# tasks.py
from celery import Celery
app = Celery(
    'tasks',
    broker='redis://localhost',
    accept_content=['pickle'],
    task_serializer='pickle'
)

@app.task
def adding(x, y):
    return x + y

if __name__ == '__main__':
    import numpy as np
    adding.apply_async((np.array([1]), np.array([1])), serializer='pickle')

此外，我还有一个配置文件：

# celeryconfig.py
print('configuring...')

accept_content = ['pickle', 'application/x-python-serialize']
task_serializer = 'pickle'
result_serializer = 'pickle'
from kombu import serialization
serialization.register_pickle()
serialization.enable_insecure_serializers()

但是，如果我运行Worker(celery -A tasks worker --loglevel=info)，然后执行进行异步调用的代码(python tasks.py)，我会得到以下回溯。我是不是遗漏了什么？

[2018-06-16 11:46:23,617: CRITICAL/MainProcess] Unrecoverable error: ContentDisallowed('Refusing to deserialize untrusted content of type pickle (application/x-python-serialize)',)
Traceback (most recent call last):
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 322, in start
    blueprint.start(self)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/consufrom celery import Celery
mer/consumer.py", line 598, in start
    c.loop(*c.loop_args())
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/loops.py", line 91, in asynloop
    next(loop)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/asynchronous/hub.py", line 354, in create_loop
    cb(*cbargs)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/transport/redis.py", line 1040, in on_readable
    self.cycle.on_readable(fileno)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/transport/redis.py", line 337, in on_readable
    chan.handlers[type]()
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/transport/redis.py", line 724, in _brpop_read
    self.connection._deliver(loads(bytes_to_str(item)), dest)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line 983, in _deliver
    callback(message)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line 633, in _callback
    return callback(message)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/messaging.py", line 624, in _receive_callback
    return on_m(message) if on_m else self.receive(decoded, message)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 572, in on_task_received
    callbacks,
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/celery/worker/strategy.py", line 136, in task_message_handler
    if body is None and 'args' not in message.payload:
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/message.py", line 207, in payload
    return self._decoded_cache if self._decoded_cache else self.decode()
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/message.py", line 192, in decode
    self._decoded_cache = self._decode()
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/message.py", line 197, in _decode
    self.content_encoding, accept=self.accept)
  File "/opt/anaconda/envs/Python3/lib/python3.6/site-packages/kombu/serialization.py", line 253, in loads
    raise self._for_untrusted_content(content_type, 'untrusted')
kombu.exceptions.ContentDisallowed: Refusing to deserialize untrusted content of type pickle (application/x-python-serialize)

推荐答案

对于回答此问题的任何人：

答案是使用app.config_from_object方法：

import celeryconfig
app.config_from_object(celeryconfig)

这篇关于芹菜即使允许腌制也不接受腌制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！