为Scikit图像构建自定义AWS Lambda层 [英] Build custom AWS Lambda layer for Scikit-image
问题描述
概述::我需要在某些AWS lambda函数中使用scikit-image，因此我希望构建一个包含scikit-image的自定义AWS lambda层.
Outline: I need to use scikit-image inside some AWS lambda functions, so I'm looking to build a custom AWS lambda layer containing scikit-image.
我的问题通常应适用于任何python模块，尤其是scikit-learn，或我认为的任何常规自定义层.
背景:经过大量的搜索和阅读，看来最好的方法是使用docker在本地运行AWS Lambda运行时，然后在其中安装/编译scikit-image(或任何一个)您要查找的模块).完成后，您可以将其作为自定义层上传/安装到AWS.
Background: After much googling and reading it seems the best way to do that is to use docker to run the AWS lambda runtime locally, and then inside there install/compile scikit-image (or whichever module you're looking for). After that's done, you can upload/install it to AWS as a custom layer.
从概念上讲这很简单，但是我在用最佳实践方法来解决这个问题上有些挣扎.我已经做到了这一点，但不确定我是否以最佳/正确/最佳/安全的方式来做... 有上百万篇关于此的博客文章，略有不同，并且AWS文档本身(IMHO)太详细了，但跳过了一些基本问题.
This is conceptually pretty simple, but I'm struggling a bit with best-practices way to do this. I've got this working, but not sure I'm doing it the best/right/optimal/secure way ... there are million all-slightly-different blog posts about this, and the AWS docs themselves are (IMHO) too detailed but skip over some of the basic questions.
我一直在尝试基本上遵循两个不错的中等职位，此处和
I've been trying to basically follow two good medium posts, here and here ...kudos to those guys.
我的主要问题是:
- 在哪里找到最新的AWS AMI docker镜像的最佳位置?
对于所谓的最新图像，有多个(甚至在亚马逊本身上)多个位置/版本等.例如 https://docs.aws.amazon.com/lambda/Latest/dg/lambda-runtimes.html 或 https://cdn .amazonlinux.com/os-images/2.0.20190823.1/.
There are multiple (even on amazon itself) multiple locations/versions etc for what is supposedly the latest image. eg https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html, or https://cdn.amazonlinux.com/os-images/2.0.20190823.1/.
..这忽略了非亚马逊github托管的众多可能性，例如中等帖子
..This is ignoring the multitude of non-amazon github hosted possibilities, such as lambci/lambda:build-python3.6 from medium posts here, or onema/amazonlinux4lambda from here.
出于安全性和最新性的考虑，我更喜欢使用Amazon提供的docker映像.
I'd prefer to use an amazon provided docker image, for both security and up-to-date'ness.
- AWS lambda运行时是否在此处，链接到
- Is the AWS lambda runtime here, which links to this AMI, a docker image? If so (or not) how do you download it to run it locally?
- How do you ensure you know when you might need to rebuild a layer, because the AWS lambda runtime is changed by amazon and that breaks you're layer using an older runtime?
- Is it better to build (compile in the case of scikit-image) the pip installed module inside of the docker AIM container, or simply just to tell pip to download the pre-built version and hope/trust it will get the compiled libs that are the best for the AMI you're running?
基本上，我在这里关注稳定性和性能.我想确保在这种情况下，针对scikit-image的已编译库对于AMI容器而言已尽可能优化.
Basically here I'm concerned about stability and performance. I'd like to ensure that the compiled libraries for scikit-image in this case are as optimized as possible for the AMI container.
- 最好只是下载并使用 AWS的SAM 可以完成所有这些操作吗?(看起来有些过分和复杂，但是看起来确实可以确保您使用的都是正确的" AMI docker容器)时间)
- 周围是否有(好的，可信赖的)预先构建的lambda层(可能使所有这些成为讨论的重点)?我看了一下，但是找不到.
- Is it better to just download and use AWS's SAM to do all of this? (looks like overkill and complicated, but it does look like it takes care of ensuring you're using the 'correct' AMI docker container all the time)
- Are there any (good, trustable) repo's of pre-built lambda layers around (that might make all this a moot point)? I looked but couldn't find any.
...感谢您的任何建议，想法和评论！
...thanks for any advice, thoughts and comments!
推荐答案
从v0.50.0开始，sam cli的直接支持构建图层.您用有关要使用哪种运行时策略的元数据来修饰AWS::Serverless::LayerVersion资源.
As of v0.50.0, the sam cli has direct support for building layers. You decorate your AWS::Serverless::LayerVersion resource with metadata about which runtime strategy to use.
MyLayer:
Type: AWS::Serverless::LayerVersion
Properties:
Description: Layer description
ContentUri: 'my_layer/'
CompatibleRuntimes:
- python3.8
Metadata:
BuildMethod: python3.8
这篇关于为Scikit图像构建自定义AWS Lambda层的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
