在AWS Lambda上运行时未实现_multiprocessing.SemLock [英] _multiprocessing.SemLock is not implemented when running on AWS Lambda

问题描述

我有一个使用multiprocessing包的简短代码，可以在我的本地计算机上正常工作.

I have a short code that uses the multiprocessing package and works fine on my local machine.

当我上传到AWS Lambda并在那里运行时，出现以下错误(已修剪stacktrace):

When I uploaded to AWS Lambda and run there, I got the following error (stacktrace trimmed):

[Errno 38] Function not implemented: OSError
Traceback (most recent call last):
  File "/var/task/recorder.py", line 41, in record
    pool = multiprocessing.Pool(10)
  File "/usr/lib64/python2.7/multiprocessing/__init__.py", line 232, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 138, in __init__
    self._setup_queues()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 234, in _setup_queues
    self._inqueue = SimpleQueue()
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 354, in __init__
    self._rlock = Lock()
  File "/usr/lib64/python2.7/multiprocessing/synchronize.py", line 147, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1)
  File "/usr/lib64/python2.7/multiprocessing/synchronize.py", line 75, in __init__
    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 38] Function not implemented

是否可能是python核心软件包的一部分未实现?我不知道我在下面运行什么，所以我无法登录那里并进行调试.

Can it be that a part of python's core packages in not implemented? I have no idea what am I running on underneath so I can't login there and debug.

有什么想法可以在Lambda上运行multiprocessing吗?

Any ideas how can I run multiprocessing on Lambda?

推荐答案

据我所知，多处理将无法在AWS Lambda上运行，因为缺少执行环境/容器/dev/shm-请参阅

As far as I can tell, multiprocessing won't work on AWS Lambda because the execution environment/container is missing /dev/shm - see https://forums.aws.amazon.com/thread.jspa?threadID=219962 (login may be required).

关于是否/何时亚马逊将对此进行更改(我能找到).我也看过其他图书馆 https://pythonhosted.org/joblib/parallel.html 会回退到/tmp(我们知道确实存在)是否找不到/dev/shm，，但这实际上并不能解决问题.

No word (that I can find) on if/when Amazon will change this. I also looked at other libraries e.g. https://pythonhosted.org/joblib/parallel.html will fallback to /tmp (which we know DOES exist) if it can't find /dev/shm, but that doesn't actually solve the problem.

这篇关于在AWS Lambda上运行时未实现_multiprocessing.SemLock的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！