无法从本地 App Engine 开发服务器访问 BigQuery [英] Unable to access BigQuery from local App Engine development server

问题描述

这是一个专门与 Python Google AppEngine 应用程序和 Google 的 BigQuery 之间的服务器到服务器授权相关的问题，但可能与其他云服务相关.

This is specifically a question relating to server to server authorisation between a python Google AppEngine app and Google's BigQuery, but could be relevant for other cloud services.

tldr;是否可以让 App Engine 本地开发服务器向远程 BigQuery 服务进行身份验证?更好的是有本地 BigQuery 吗?

我知道 AppAssertionCredentials 目前无法在本地开发服务器上运行，尽管这本身就非常令人沮丧.

I understand that AppAssertionCredentials does not currently work on the local development server, though that in itself is very frustrating.

适用于标准 python 代码的替代方法，在本地开发服务器沙箱之外，详细此处 不适用于本地开发服务器，因为即使启用了 PyCrypto，沙箱也不允许某些 posix 模块，例如'密码'.

The alternative method which works for standard python code, outside of the local development server sandbox, detailed here does not work for the local development server because even with PyCrypto enabled the sandbox does not allow some posix modules e.g. 'pwd'.

我有 AppAssertionCredentials 在远程服务器上工作，SignedJwtAssertionCredentials 方法在本地本地 python 中工作，因此服务帐户设置正确.

I have got AppAssertionCredentials working on the remote server and the SignedJwtAssertionCredentials method working in native python locally, so the service accounts are set up properly.

导入在 try/except 块中的 oauth2client/crypt.py 中失败 - 将它们注释掉后，很容易看到沙盒白名单异常.

The imports fail within oauth2client/crypt.py within the try/except blocks - after commenting them out the sandbox whitelist exceptions are easily seen.

我在白名单中添加了pwd"，然后又出现了另一个问题，所以我赶紧从那个兔子洞里跑了出来.

I've fiddled around with adding 'pwd' to the whitelist, then another problem crops up, so I scurried back out of that rabbit hole.

我试过将 PyCrypto 直接包含到项目中，结果相似.

I've tried including PyCrypto directly into the project with similar results.

我也尝试过使用 OpenSSL，结果相似.

I've also tried with OpenSSL with similar results.

我已经寻找了一个本地应用引擎特定的 PyCrypto 无济于事，我错过了一个吗?我应该说这是在 Mac OSX 上 - 也许我应该启动一个 linux 机器并试一试?

I have looked for a local appengine specific PyCrypto to no avail, have I missed one? I should say this is on Mac OSX - perhaps I should fire up a linux box and give that a go?

推荐答案

最近发布的 Google App Engine SDK 添加了对开发服务器上的 AppAssertionCredentials 方法的支持.要在本地使用此方法，请将以下参数添加到 dev_appserver.py:

A recent release of Google App Engine SDK added support for the AppAssertionCredentials method on the development server. To use this method locally, add the following arguments to dev_appserver.py:

$ dev_appserver.py --help
...
Application Identity:
  --appidentity_email_address APPIDENTITY_EMAIL_ADDRESS
                        email address associated with a service account that
                        has a downloadable key. May be None for no local
                        application identity. (default: None)
  --appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH
                        path to private key file associated with service
                        account (.pem format). Must be set if
                        appidentity_email_address is set. (default: None)

要使用这些:

1. 在 Google Developer Console 中，选择一个项目，然后导航至API &"授权"->凭据"->创建新的客户端 ID".

1. In Google Developer Console, select a project then navigate to "API & auth" -> "Credentials" -> "Create new client ID".

选择服务帐户"；并按照提示下载 PKCS12 (.p12) 格式的私钥.记下服务帐户的电子邮件地址.

Select "Service account" and follow the prompts to download the private key in PKCS12 (.p12) format. Take note of the email address for the service account.

确保您将该服务帐户电子邮件地址添加到权限"包含需要访问的数据的任何项目的选项卡，默认情况下，它会添加到创建它的项目团队中.

Make sure you add that service account email address to the "Permissions" tab for any project that contains data it needs to access, by default it is added to the project team in which it was created.

使用以下命令将 PKCS12 格式转换为 PKCS1 格式:

Convert the PKCS12 format to PKCS1 format using the following command:

$ cat/path/to/xxxx-privatekey.p12 |openssl pkcs12 -nodes -nocerts -passin pass:notasecret |openssl rsa >/path/to/secret.pem

启动 dev_appserver.py 为:

$ dev_appserver.py --appidentity_email_address [e-mail] --appidentity_private_key_path/path/to/secret.pem ...

以与在生产中通常使用的方式相同的方式在本地使用 appidentity 模块和 AppAssertionCredentials.

Use appidentity module and AppAssertionCredentials in the same manner locally as you normally would in production.

请确保 /path/to/secret.pem 在您的应用程序源目录之外，以免意外将其部署为应用程序的一部分.

Please ensure that /path/to/secret.pem is outside of your application source directory so that it is not accidentally deployed as part of your application.

这篇关于无法从本地 App Engine 开发服务器访问 BigQuery的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！