无法在AWS Lambda上导入LXML etree [英] Unable to import lxml etree on aws lambda
问题描述
{
"errorMessage": "Unable to import module 'lambda_function':
cannot import name 'etree' from 'lxml' (/var/task/lxml/__init__.py)",
"errorType": "Runtime.ImportModuleError"
}
还尝试了 https://gist.github.com/allen-munsch/ad8faf9c04b72aa8d0808fa8953bcc639:
{
"errorMessage": "Unable to import module 'lambda_function':
cannot import name 'etree' from 'lxml'
(/var/task/lxml-4.3.4-py3.6-linux-x86_64.egg/lxml/__init__.py)",
"errorType": "Runtime.ImportModuleError"
}
我正在本地计算机上的Ubuntu 18.04上运行，并且还尝试在ec2实例上使用"Amazon Linux"映像来构建捆绑包.
I am running on Ubuntu 18.04 on my local machine, and have also tried using the "Amazon Linux" image on an ec2 instance to build the bundle.
在激活的venv中，我也尝试过:
I have also tried, while within the activated venv:
STATIC_DEPS=true pip3 install lxml --target ./package --upgrade --no-cache-dir
我还尝试根据使用strace运行脚本时拉开了哪些文件来复制共享对象文件:
I have also tried copying the shared object files based on pulling which files were opened when running the script with strace:
#! /bin/bash
export Z=$(pwd)/ok-daily-lambda.zip
rm $Z
zip $Z lambda_function.py
zip $Z __init__.py
for dir in $(find venv_here/lib/python3.6/site-packages)
do
if [ -d $dir ] ; then
pushd $dir
echo zip -r9 $Z $(pwd)
zip -r9 $Z $(pwd)
popd
fi
done
export LIBD=$(pwd)/lib
mkdir -p $LIBD
cp "/home/jmunsch/.local/lib/python3.6/site-packages/.libs_cffi_backend/libffi-d78936b1.so.6.0.4" $LIBD
cp "/lib/x86_64-linux-gnu/libbz2.so.1.0" $LIBD
cp "/lib/x86_64-linux-gnu/libc.so.6" $LIBD
cp "/lib/x86_64-linux-gnu/libdl.so.2" $LIBD
cp "/lib/x86_64-linux-gnu/libexpat.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/libgcc_s.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/liblzma.so.5" $LIBD
cp "/lib/x86_64-linux-gnu/libm.so.6" $LIBD
cp "/lib/x86_64-linux-gnu/libnss_dns.so.2" $LIBD
cp "/lib/x86_64-linux-gnu/libnss_files.so.2" $LIBD
cp "/lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2" $LIBD
cp "/lib/x86_64-linux-gnu/libpthread.so.0" $LIBD
cp "/lib/x86_64-linux-gnu/libresolv.so.2" $LIBD
cp "/lib/x86_64-linux-gnu/librt.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/libtinfo.so.5" $LIBD
cp "/lib/x86_64-linux-gnu/libudev.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/libutil.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/libuuid.so.1" $LIBD
cp "/lib/x86_64-linux-gnu/libz.so.1" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libffi.so.6" $LIBD
cp "/usr/lib/x86_64-linux-gnu/liblz4.so.1" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libmpdec.so.2" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libssl.so.1.1" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libstdc++.so.6" $LIBD
cp "/usr/lib/x86_64-linux-gnu/libzstd.so.1" $LIBD
zip -r $Z $LIBD
AWS_ACCESS_KEY_ID="xxx" AWS_SECRET_ACCESS_KEY="xxx" AWS_DEFAULT_REGION="us-east-1" aws lambda update-function-code --function-name ok-today --zip-file fileb://ok-daily-lambda.zip
这是最新的zip文件的目录结构:
Here is the directory structure of the most recent zip file:
相关:
- AWS Lambda未导入LXML
- LXML:无法导入etree
- https://lxml.de/installation.html
- https://lxml.de/build.html
- lxml无法导入元素etree
- Java AWS Lambda项目中的共享库
- https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
- AWS Lambda not importing LXML
- LXML: Cannot import etree
- https://lxml.de/installation.html
- https://lxml.de/build.html
- lxml cannot import element etree
- Shared Libraries in Java AWS Lambda Project
- https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
推荐答案
有些模块无法直接添加到site-packages目录中，从而无法在AWS Lambda环境中识别.发生这种情况时，您必须从Docker存储库中获取Amazon Linux映像，并在将在AWS Lambda上运行的容器版本中创建自己的编译环境
There are modules that cannot be added directly into the site-packages directory to be recognised inside an AWS Lambda environment. When that happens, you have to get an Amazon Linux image from Docker repositories and make your own compiled environment in a container version that will run on AWS Lambda
例如，如果您想使用Python 3.6，最好选择amazonlinux:2018.03，以防您要安装更多软件包，例如熊猫，numpy，scipy
For example, if you want to use Python 3.6 a good choice will be amazonlinux:2018.03 in case you want to install more packages e.g. pandas, numpy, scipy
docker run -v $(pwd):/outputs -it amazonlinux:2018.03
由于Amazon Linux基于Red Hat，因此在运行docker并已创建虚拟环境时，您必须通过yum所有依赖项进行安装
Since Amazon Linux is based on Red Hat, you have to install via yum all dependencies when running docker and having already created your virtual environment
yum update -y
yum install -y \
python36 \
python36-devel \
python36-virtualenv \
python34-setuptools \
gcc \
gcc-c++ \
findutils \
rsync \
Cython \
findutils \
which \
gzip \
tar \
man-pages \
man \
wget \
make \
zip
对于lxml，您还需要
(lambda_docker) bash-4.2# yum install libxml2
...
(lambda_docker) bash-4.2# yum install libxslt
...
您照常安装模块
pip3.6 install lxml
您应该会看到类似的东西
You should see something like
(lambda_docker) bash-4.2# pip3.6 install lxml
Collecting lxml
Downloading https://files.pythonhosted.org/packages/2d/53/34a9f0c79c548e430148837892b6ae91adee571a0e8b6c17bd7ff9c2d12e/lxml-4.3.4-cp36-cp36m-manylinux1_x86_64.whl (5.7MB)
|################################| 5.7MB 2.0MB/s
Installing collected packages: lxml
然后，创建您的函数lambda_function.py并将其添加到您从docker session推入和弹出的压缩包中
Then, create your function lambda_function.py and add it in the zipped package pushd-ed and popd-ed from you docker session
from lxml import etree
def lambda_handler(event, context):
print(__name__)
print(etree.LXML_VERSION)
创建后
zip -g site-packages.zip lambda_function.py
在上传之前，您可以检查zip文件中是否包含lxml目录
Before uploading, you can check that your zip file contains the lxml directories
[jonathan@docker lambda_docker]$ unzip -l site-packages.zip
Archive: site-packages.zip
Length Date Time Name
--------- ---------- ----- ----
0 06-29-2019 23:09 __pycache__/
277 06-29-2019 23:09 __pycache__/easy_install.cpython-36.pyc
126 06-29-2019 23:09 easy_install.py
119 06-29-2019 23:29 lambda_function.py
0 06-29-2019 23:21 lib/
0 06-29-2019 23:39 lxml/
0 06-29-2019 23:37 lxml-4.3.4.dist-info/
4 06-29-2019 23:37 lxml-4.3.4.dist-info/INSTALLER
2954 06-29-2019 23:37 lxml-4.3.4.dist-info/METADATA
13384 06-29-2019 23:37 lxml-4.3.4.dist-info/RECORD
109 06-29-2019 23:37 lxml-4.3.4.dist-info/WHEEL
5 06-29-2019 23:37 lxml-4.3.4.dist-info/top_level.txt
7668 06-29-2019 23:37 lxml/ElementInclude.py
551 06-29-2019 23:37 lxml/__init__.py
0 06-29-2019 23:37 lxml/__pycache__/
3331 06-29-2019 23:37 lxml/__pycache__/ElementInclude.cpython-36.pyc
...
现在上传zip并在您的lambda函数中创建一个空测试{}
Now upload the zip and create an empty test {} in your lambda function
结果
START RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64 Version: $LATEST
lambda_function
(4, 3, 4, 0)
END RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64
REPORT RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64 Duration: 0.30 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 50 MB
如果您喜欢图片
完全可以使用AWS Lambda
Perfectly ready for AWS Lambda
希望它会有所帮助(:
这篇关于无法在AWS Lambda上导入LXML etree的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
