无法在AWS Lambda层中导入 pandas [英] Unable to import Pandas in AWS Lambda layer

问题描述

我将python大熊猫上载到Lambda，当我运行Lambda时，看到以下错误:

I uploaded python pandas to Lambda and when I run the Lambda, I see the following error:

"errorMessage": "Unable to import module 'lambda_function': C extension: No module named 'pandas._libs.interval' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first."

我真的不确定这里应该发生什么.但是为了获得更多的上下文，我创建了一个名为"python"的目录.然后在终端上运行以下行

I am really not sure what is supposed to be going on here. But for more context, I created a directory called "python" and then I run the following line on my terminal

python3.8 -m pip install pandas -t .

然后我将"python"文件解压缩.目录，然后创建一个新层并上传zip文件.

I then zip the "python" directory and then create a new layer and upload the zip file.

努力看待我的短消息.

推荐答案

您是否考虑过对 pandas 使用公开可用的图层，例如从以下流行的存储库中获取:

Have you considered using publicly available layer for pandas, for example from this popular repository:keithrozario/Klayers?

例如，可用于python 3.8的层列表为

For instance, the list of layers available for python 3.8 is here for us-east-1.

在这种情况下，对于 us-east-1 ，您可以使用以下方法添加 pandas 层:

In this case, for us-east-1 you could add the pandas layer using:

arn:aws:lambda:us-east-1:770693421928:layer:Klayers-python38-pandas:16

更新，创建自定义图层

我刚刚使用 pandas 和 xlrd 创建了自定义图层，并可以确认其正常工作.

Update, making custom layer

I just created the custom layer with pandas and xlrd and can confirm that it works.

所使用的技术包括最近的 AWS博客中所述的 docker工具:

The technique used includes docker tool described in the recent AWS blog:

• 如何使用模拟Lambda创建Lambda层Docker环境?

因此，对于这个问题，我验证了，如下所示:

Thus for this question, I verified it as follows:

1. 创建空文件夹，例如 mylayer .

转到该文件夹​​并创建内容为

的 requirements.txt 文件

Go to the folder and create requirements.txt file with the content of

pandas
xlrd

1. 运行以下docker命令:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"

1. 将层创建为zip:

zip -r mypandaslayer.zip python > /dev/null

1. 在AWS Console中基于 mypandaslayer.zip 创建lambda层.不要忘记为 python3.8 指定兼容运行时.

1. Create lambda layer based on mypandaslayer.zip in the AWS Console. Don't forget to specify Compatible runtimes to python3.8.

使用以下lambda函数在lambda中测试图层:

Test the layer in lambda using the following lambda function:

import json

import pandas
import xlrd

def lambda_handler(event, context):
    
    print(dir(pandas))
    print(dir(xlrd))

该函数正确执行:

['BooleanDtype', 'Categorical', 'CategoricalDtype', 'CategoricalIndex', 'DataFrame', 'DateOffset', 'DatetimeIndex', 'DatetimeTZDtype', 'ExcelFile', 'ExcelWriter', 'Float64Index', 'Grouper', 'HDFStore', 'Index', 'IndexSlice', 'Int16Dtype', 'Int32Dtype', 'Int64Dtype', 'Int64Index', 'Int8Dtype', 'Interval', 'IntervalDtype', 'IntervalIndex', 'MultiIndex', 'NA', 'NaT', 'NamedAgg', 'Period', 'PeriodDtype', 'PeriodIndex', 'RangeIndex', 'Series', 'SparseDtype', 'StringDtype', 'Timedelta', 'TimedeltaIndex', 'Timestamp', 'UInt16Dtype', 'UInt32Dtype', 'UInt64Dtype', 'UInt64Index', 'UInt8Dtype', '__builtins__', '__cached__', '__doc__', '__docformat__', '__file__', '__getattr__', '__git_version__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_config', '_hashtable', '_is_numpy_dev', '_lib', '_libs', '_np_version_under1p16', '_np_version_under1p17', '_np_version_under1p18', '_testing', '_tslib', '_typing', '_version', 'api', 'array', 'arrays', 'bdate_range', 'compat', 'concat', 'core', 'crosstab', 'cut', 'date_range', 'describe_option', 'errors', 'eval', 'factorize', 'get_dummies', 'get_option', 'infer_freq', 'interval_range', 'io', 'isna', 'isnull', 'json_normalize', 'lreshape', 'melt', 'merge', 'merge_asof', 'merge_ordered', 'notna', 'notnull', 'offsets', 'option_context', 'options', 'pandas', 'period_range', 'pivot', 'pivot_table', 'plotting', 'qcut', 'read_clipboard', 'read_csv', 'read_excel', 'read_feather', 'read_fwf', 'read_gbq', 'read_hdf', 'read_html', 'read_json', 'read_orc', 'read_parquet', 'read_pickle', 'read_sas', 'read_spss', 'read_sql', 'read_sql_query', 'read_sql_table', 'read_stata', 'read_table', 'reset_option', 'set_eng_float_format', 'set_option', 'show_versions', 'test', 'testing', 'timedelta_range', 'to_datetime', 'to_numeric', 'to_pickle', 'to_timedelta', 'tseries', 'unique', 'util', 'value_counts', 'wide_to_long']
['Book', 'FMLA_TYPE_ARRAY', 'FMLA_TYPE_CELL', 'FMLA_TYPE_COND_FMT', 'FMLA_TYPE_DATA_VAL', 'FMLA_TYPE_NAME', 'FMLA_TYPE_SHARED', 'MMAP_AVAILABLE', 'Operand', 'Ref3D', 'USE_MMAP', 'X12Book', 'XLDateError', 'XLRDError', 'XL_CELL_BLANK', 'XL_CELL_BOOLEAN', 'XL_CELL_DATE', 'XL_CELL_EMPTY', 'XL_CELL_ERROR', 'XL_CELL_NUMBER', 'XL_CELL_TEXT', '__VERSION__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 'biff_text_from_num', 'biffh', 'book', 'cellname', 'cellnameabs', 'colname', 'compdoc', 'count_records', 'decompile_formula', 'dump', 'dump_formula', 'empty_cell', 'error_text_from_code', 'evaluate_name_formula', 'formatting', 'formula', 'info', 'mmap', 'oBOOL', 'oERR', 'oNUM', 'oREF', 'oREL', 'oSTRG', 'oUNK', 'okind_dict', 'open_workbook', 'os', 'pprint', 'rangename3d', 'rangename3drel', 'sheet', 'sys', 'timemachine', 'xldate', 'xldate_as_datetime', 'xldate_as_tuple', 'xlsx', 'zipfile']

这篇关于无法在AWS Lambda层中导入 pandas 的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！