适用于 Python 和 AWS Lambda 的 Google Cloud Platform API 不兼容:无法导入名称“cygrpc" [英] Google Cloud Platform API for Python and AWS Lambda Incompatibility: Cannot import name 'cygrpc'

问题描述

我正在尝试将 Google Cloud Platform(特别是 Vision API)与 AWS Lambda 一起用于 Python.因此，我必须为我的依赖项创建一个部署包.但是，当我尝试创建此部署包时，无论 Python 版本是什么(3.6 或 2.7)，我都会遇到几个编译错误.考虑到 3.6 版，我收到了无法导入名称‘cygrpc’"的问题.对于 2.7，我在 .path 文件中遇到一些未知错误.我正在遵循 AWS Lambda 部署包说明 此处.他们推荐了两个选项，但都不起作用/导致相同的问题.由于某种原因，GCP 是否与 AWS Lambda 不兼容?怎么回事?

I am trying to use Google Cloud Platform (specifically, the Vision API) for Python with AWS Lambda. Thus, I have to create a deployment package for my dependencies. However, when I try to create this deployment package, I get several compilation errors, regardless of the version of Python (3.6 or 2.7). Considering the version 3.6, I get the issue "Cannot import name 'cygrpc'". For 2.7, I get some unknown error with the .path file. I am following the AWS Lambda Deployment Package instructions here. They recommend two options, and both do not work / result in the same issue. Is GCP just not compatible with AWS Lambda for some reason? What's the deal?

Python 3.6 和 2.7 都不适合我.

Neither Python 3.6 nor 2.7 work for me.

注意:我在这里发布这个问题是为了自己回答，因为我花了很长时间才找到解决方案，我想分享我的解决方案.

NOTE: I am posting this question here to answer it myself because it took me quite a while to find a solution, and I would like to share my solution.

推荐答案

TL;DR:您无法在 Mac 或您使用的任何 PC 上编译部署包.您必须使用特定的操作系统/设置"来执行此操作，这与 AWS Lambda 用于运行您的代码的操作系统相同.为此，您必须使用 EC2.

TL;DR: You cannot compile the deployment package on your Mac or whatever pc you use. You have to do it using a specific OS/"setup", the same one that AWS Lambda uses to run your code. To do this, you have to use EC2.

我将在此处提供有关如何让 Google Cloud Vision 在适用于 Python 2.7 的 AWS Lambda 上工作的答案.此答案可能适用于 AWS Lambda 上的其他 API 和其他编程语言.

I will provide here an answer on how to get Google Cloud Vision working on AWS Lambda for Python 2.7. This answer is potentially extendable for other other APIs and other programming languages on AWS Lambda.

所以我的解决方案之旅始于这篇最初的帖子 与其他有同样问题的人一起在 Github 上.有人发布的一种解决方案是

So the my journey to a solution began with this initial posting on Github with others who have the same issue. One solution someone posted was

我在运行时遇到了同样的问题无法导入名称‘cygrpc’"拉姆达.在 AMI 中使用 pip install google-cloud-vision 解决了它amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 实例并导出lib/python3.6/site-packages 到 aws lambda 谢谢@tseaver

I had the same issue " cannot import name 'cygrpc' " while running the lambda. Solved it with pip install google-cloud-vision in the AMI amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 instance and exported the lib/python3.6/site-packages to aws lambda Thank you @tseaver

这部分是正确的，除非我读错了，但无论如何它使我走上了正确的道路.您将不得不使用 EC2.以下是我采取的步骤:

This is partially correct, unless I read it wrong, but regardless it led me on the right path. You will have to use EC2. Here are the steps I took:

1. 通过访问 Amazon 上的 EC2 来设置 EC2 实例.如果您还没有，请快速阅读一下 AWS EC2.为 amzn-ami-hvm-2018.03.0.20180811-x86_64-gp2 或类似的东西设置一个(即最新的).

2. 获取您的 EC2 .pem 文件.前往您的终端.cd 进入您的 .pem 文件所在的文件夹.使用

1. Set up an EC2 instance by going to EC2 on Amazon. Do a quick read about AWS EC2 if you have not already. Set one up for amzn-ami-hvm-2018.03.0.20180811-x86_64-gp2 or something along those lines (i.e. the most updated one).

2. Get your EC2 .pem file. Go to your Terminal. cd into your folder where your .pem file is. ssh into your instance using

ssh -i "your-file-name-here.pem" [e-mail]

ssh -i "your-file-name-here.pem" [e-mail]

使用 mkdir 在您的实例上创建以下文件夹:google-cloud-vision、protobuf、google-api-python-client、httplib2、uritemplate、google-auth-httplib2.

Create the following folders on your instance using mkdir: google-cloud-vision, protobuf, google-api-python-client, httplib2, uritemplate, google-auth-httplib2.

在您的 EC2 实例上，cd 进入 google-cloud-vision.运行命令:

On your EC2 instance, cd into google-cloud-vision. Run the command:

pip install google-cloud-vision -t .

pip install google-cloud-vision -t .

注意如果你得到bash: pip: command not found"，那么输入sudo easy_install pip"​​来源.

Note If you get "bash: pip: command not found", then enter "sudo easy_install pip" source.

1. 使用以下软件包重复第 4 步，同时 cd 到相应的文件夹中:protobuf、google-api-python-client、httplib2、uritemplate、google-auth-httplib2.

1. Repeat step 4 with the following packages, while cd'ing into the respective folder: protobuf, google-api-python-client, httplib2, uritemplate, google-auth-httplib2.

复制您计算机上的每个文件夹.您可以使用 scp 命令执行此操作.同样，在您的终端，而不是您的 EC2 实例和您用来访问 EC2 实例的终端窗口中，运行命令(以下是google-cloud-vision"文件夹的示例，但对每个文件夹重复此操作):

Copy each folder on your computer. You can do this using the scp command. Again, in your Terminal, not your EC2 instance and not the Terminal window you used to access your EC2 instance, run the command (below is an example for your "google-cloud-vision" folder, but repeat this with every folder):

sudo scp -r -i your-pem-file-name.pem [e-mail]:~/google-cloud-vision ~/Documents/your-本地目录/

sudo scp -r -i your-pem-file-name.pem [e-mail]:~/google-cloud-vision ~/Documents/your-local-directory/

从 AWS 控制台停止您的 EC2 实例，这样您就不会被多收费用.

Stop your EC2 instance from the AWS console so you don't get overcharged.

对于您的部署包，您需要一个包含所有模块和 Python 脚本的文件夹.要开始组合所有模块，请创建一个名为模块"的空文件夹.将google-cloud-vision"文件夹的所有内容复制并粘贴到modules"文件夹中.现在只将protobuf"(原文如此)主文件夹中名为protobuf"的文件夹放入modules"文件夹的Google"文件夹中.同样从protobuf"主文件夹中，将 Protobuf .pth 文件和 -info 文件夹粘贴到 Google 文件夹中.

For your deployment package, you will need a single folder containing all your modules and your Python scripts. To begin combining all of the modules, create an empty folder titled "modules." Copy and paste all of the contents of the "google-cloud-vision" folder into the "modules" folder. Now place only the folder titled "protobuf" from the "protobuf" (sic) main folder in the "Google" folder of the "modules" folder. Also from the "protobuf" main folder, paste the Protobuf .pth file and the -info folder in the Google folder.

对于 protobuf 之后的每个模块，将标题为模块名称的文件夹、.pth 文件和-info"文件夹复制并粘贴到模块"文件夹中.

For each module after protobuf, copy and paste in the "modules" folder the folder titled with the module name, the .pth file, and the "-info" folder.

您现在已经正确组合了所有模块(几乎).要完成组合，请从模块"文件夹中删除这两个文件:googleapis_common_protos-1.5.3-nspkg.pth 和 google_cloud_vision-0.34.0-py3.6-nspkg.pth.将modules"文件夹中的所有内容复制并粘贴到您的部署包文件夹中.此外，如果您使用 GCP，请粘贴您的 .json 文件作为您的凭据.

You now have all of your modules properly combined (almost). To finish combination, remove these two files from your "modules" folder: googleapis_common_protos-1.5.3-nspkg.pth and google_cloud_vision-0.34.0-py3.6-nspkg.pth. Copy and paste everything in the "modules" folder into your deployment package folder. Also, if you're using GCP, paste in your .json file for your credentials as well.

最后，将您的 Python 脚本放在此文件夹中，压缩内容(不是文件夹)，上传到 S3，然后将链接粘贴到您的 AWS Lambda 函数中，然后开始！

Finally, put your Python scripts in this folder, zip the contents (not the folder), upload to S3, and paste the link in your AWS Lambda function and get going!

如果这里的某些内容与描述的不一样，请原谅我并向我发送消息或随时编辑我的答案.希望这会有所帮助.

If something here doesn't work as described, please forgive me and either message me or feel free to edit my answer. Hope this helps.

这篇关于适用于 Python 和 AWS Lambda 的 Google Cloud Platform API 不兼容:无法导入名称“cygrpc"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！