Python限制中的AWS Lambda部署软件包 [英] AWS Lambda Deployment Package in Python Limites

问题描述

我想在AWS lambda函数上运行我的代码.为此，我需要导入一些python软件包(即pandas，numpy，sklearn，scipy)

I want to run my code on AWS lambda function. To do so, i need to import some python packages (i.e. pandas, numpy, sklearn, scipy)

我有两个问题: 首先，(解压缩)打包的python zip文件的大小大于250MB.

I have two problems: First of all, the size of (unzip) packaged python zip files is greater than 250MB.

第二，我在使用scipy和sklearn时遇到了一些错误:

Secondly, I got some error using scipy as well as sklearn as:

无法导入模块"lambda_function":无法导入名称 '_ccallback_c'

Unable to import module 'lambda_function': cannot import name '_ccallback_c'

的

无法导入模块"lambda_function":未命名模块 'sklearn. check_build._check_build' ___________________________________________________________________________/var/task/sklearn/__ check_build的内容: __pycache _check_build.cpython-35m-x86_64-linux-gnu.sosetup.py 初始化 .py ___________________________________________________________________________似乎scikit-learn的构建不正确.

Unable to import module 'lambda_function': No module named 'sklearn.check_build._check_build' ___________________________________________________________________________ Contents of /var/task/sklearn/__check_build: __pycache _check_build.cpython-35m-x86_64-linux-gnu.sosetup.py init.py ___________________________________________________________________________ It seems that scikit-learn has not been built correctly.

我尝试重新安装多次... 但是sklearn和scipy仍然存在问题.

I tried to reinstall many times... But still problems in sklearn and scipy.

有什么主意吗?

AWS LambdaFunction中的示例代码:

sample code in AWS LambdaFunction:

import json
import numpy
import pandas
import sklearn
import scipy 


def lambda_handler(event, context):
    # TODO implement
    print(event)
    return

推荐答案

您似乎有两个问题.

第一个(也是最容易解决的)是您需要在与Amazon Linux相当的Linux发行版上安装相关模块.

The first (and easiest to solve) is that you need to install the relevant modules on a Linux distro comparable to Amazon Linux.

您可以使用EC2或在装有Amazon Linux的Docker容器中执行此操作.

You can either do this using EC2 or in a Docker container with Amazon Linux on it.

第二个问题(考虑到要使用的模块的大小，如果不是不可能解决的话，这将比较棘手)是您需要将部署大小压缩到250MB以下，压缩到50MB以下.

The second issue (which is a bit trickier if not impossible to solve given the size of the modules you want to use) is that you need to get your deployment size down to under 250MB unzipped and under 50MB zipped.

在安装时使用相关的CFLAG可能会为您提供一些帮助.请参见此处了解可能有效的方法.

Using relevant CFLAG when installing may get you some of the way there. See here for an idea of what might work.

如果您仍然超出限制(我怀疑您会成为限制)，那么您唯一的选择就是删除一些模块中您认为不会在特定程序中使用的文件.这是有风险的，通常容易出错，并且通常需要进行很多尝试才能正确.使用代码覆盖率工具可能会对您有所帮助，因为它们可以指示实际正在使用的文件.

If you are still over limit (which I suspect you will be) your only choice left will be to delete some of the files in the modules which you believe will not be used in your particular program. This is risky, often error prone and usually takes many attempts to get right. Using code coverage tools may help you here, as they can indicate which files are actually being used.

这篇关于Python限制中的AWS Lambda部署软件包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！