AWS Lambda，Python，Numpy等作为图层 [英] AWS Lambda, Python, Numpy and others as Layers

问题描述

我已经花了一段时间尝试将python，numpy和pytz作为Layers添加到AWS Lambda中，而不必压缩并使用.py文件将其扔到AWS上.我能够遵循多个教程，但所有教程都失败了.

I have been going at this for a while trying to get python, numpy and pytz added to AWS Lambda as Layers rather than having to zip and throw it at AWS with my .py file. I was able to follow multiple tutorials and all of them failed.

如果我要使用熊猫，numpy或pytz的任何功能，则必须遵循本指南(带熊猫和NumPy的AWS Lambda-Ruslan Korniichuk-Medium ).因此，这很好，但是如果事情因我的功能需要而改变，我不想每次都重新创建一个zip.特别是随着我公司的发展.我们只是尝试使用Cloudwatch使用Lamba自动执行一些任务，以定期运行作业.没什么特别的，我知道可能会有S3和其他实例的路线.但是，我已经能够为Pandas，Numpy和Pytz之外的其他库成功创建图层.

I have resorted to following this guide if I am to go with pandas, numpy or pytz for any functionality (AWS Lambda with Pandas and NumPy - Ruslan Korniichuk - Medium). So this is good but I do not want to have to recreate a zip each time if things change with my function needs ect. especially as my company is growing. We are simply trying to automate some tasks with Lamba using Cloudwatch to run jobs periodically. Nothing spectacular and I know there may be route with S3 and other instances. However, I have been able to successfully create layers for other libraries except for Pandas, Numpy and Pytz.

因此，我担心这种方法的可扩展性.我在Mac上工作，但不确定要做什么:我尝试使用Docker，尝试从轮子构建.有没有可行的教程来详细说明如何执行此操作?

So, I am worried about scalability with this method. I am working on a mac and I am not sure what else to do: I have tried using Docker, I have tried building from wheels. Is there any viable tutorials that explain how to do this in detail?

这是我尝试过的一些教程.是的，这并不意味着我正确地遵循了它们，但是最后大多数都没有成功:

Here are some of the tutorials I have tried. Yes, it does not mean I followed them correctly but I did not succeed in the end with most of them:

• AWS Lambda与熊猫和NumPy-Ruslan Korniichuk-中
• Amazon简介使用Python3的Lambda，Lays和boto3
• 创建新的AWS LambdaPython Pandas库图层

您为它命名，我可能已经完成了这些步骤，尤其是在这些文章中，以完成此任务.还有很多Stack问题注释，这些注释非常有帮助且很有见地.

You name it and I might have already went through the steps, especially in these articles, to complete this task. And a lot of Stack question comments as well, which have been very helpful and insightful.

在此先感谢您提供任何建议！

Thanks in advance for any advice, just here to learn!

推荐答案

这可能不是您想要听到的答案，但是说实话，将某些编译的库放入lambda层所带来的痛苦足以使我的公司停止使用它们.取而代之的是，我们倾向于将fargate或ECS与docker容器一起使用.

This is probably not the answer you want to hear, but honestly the pain around getting certain compiled libraries into lambda layers was enough for my company to just stop using them. Instead we use tend to use either fargate or ECS with docker containers.

除了为lambda编译软件包的问题外，我们还遇到了有关lambda的最大大小的主要问题.我们通常会达到这个上限，并且为了使文件适合，我们不得不越来越多地删除文件.

Besides the issues of compiling packages for lambdas, we also ran into major issues with the max size of lambdas. We regularly were hitting that cap and having to get more and more hacky to remove files in order to make them fit.

更新:AWS现在允许您从ECR中的容器运行Lambda，这很好地解决了这个问题.

Update: AWS now lets you run Lambdas from containers in ECR, which solves this problem nicely.

这篇关于AWS Lambda，Python，Numpy等作为图层的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！