从Python 3.8中的AWS lambda temp目录导入软件包 [英] Importing a package from AWS lambda temp directory in Python 3.8

问题描述

我遇到的问题类似于 SO问题但答案对我不起作用.我正在尝试从AWS Lambda的/tmp 文件夹导入Python库(假设为 xgboost ).

The problem I have is similar to this SO question but the answer doesn't work for me. I am trying to import Python library (let's say xgboost) from /tmp folder in AWS Lambda.

库 requests 已添加到Lambda层，我所做的是:

Library requests is added to Lambda layer and what I did is:

import json
import io
import os
import zipfile
import requests
import sys

sys.path.insert(0, '/tmp/')
sys.path.append('/tmp/')

os.environ["PYTHONPATH"] = "/var/task"

def get_pkgs(url):
    print("Getting Packages...")
    re = requests.get(url)
    z = zipfile.ZipFile(io.BytesIO(re.content))
    print("Extracting Packages...")
    z.extractall("/tmp/")
    print("Packages are downloaded and extracted.")
    
def attempt_import():
    print("="*50)
    print("ATTEMPT TO IMPORT DEPENDENCIES...")
    print("="*50)
    import xgboost
    print("IMPORTING DONE.")
    
def main():
    URL = "https://MY_BUCKET.s3.MY_REGION.amazonaws.com/MY_FOLDER/xgboost/xgboost.zip"

    get_pkgs(URL)
    attempt_import()
    
def lambda_handler(event, context):
    main()
    return "Hello Lambda"

我得到的错误是 [ERROR] ModuleNotFoundError:没有名为"xgboost"的模块.我授予了S3存储桶所有必要的权限，并且我肯定Lambda可以访问 .zip 文件，因为 requests.get 有效并且变量 z 返回:

The error I get is [ERROR] ModuleNotFoundError: No module named 'xgboost'. I gave my S3 bucket all necessary permissions, and I am positive that Lambda can access the .zip file since the requests.get works and variable z returns:

< zipfile.ZipFile file =< _io.BytesIO对象，位于0x7fddaf31c400>mode ='r'>

推荐答案

实际上，我上面的代码有效，并且我犯了一个非常愚蠢的错误.我实际上没有压缩 xgboost 软件包文件夹( xgboost ， xgboost.libs 和 xgboost.dist-info )压缩了他们的父母文件夹，我将其命名为 package-xgoboost ，但该文件夹在AWS lambda中不起作用.确保您实际上直接将这三个文件夹压缩了.

Actually, my code above works and I had a rather silly error. Instead of zipping the xgboost package folders (xgboost, xgboost.libs and xgboost.dist-info) I actually zipped their parental folder which I named package-xgoboost, and that didn't worked in AWS lambda. Be sure that you actually zip those 3 folders directly.

还要确保您的 xgboost 库是最新的.以前，我使用的是1.2.1版本，该版本也不起作用.最终，升级库并压缩最新的 xgboost 版本(在我的情况下为1.3.3)即可.

Also, make sure your xgboost library is up-to-date. Previously I used version 1.2.1 which didn't work either. Upgrading the library and zipping the newest xgboost version (in my case 1.3.3) finally worked.

这篇关于从Python 3.8中的AWS lambda temp目录导入软件包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！