无法从本地App Engine开发服务器访问BigQuery [英] Unable to access BigQuery from local App Engine development server

问题描述

这是一个与python Google AppEngine应用程序和Google BigQuery之间的服务器到服务器授权有关的问题，但可能与其他云服务有关。

$ b tldr;是否有可能让App Engine本地开发服务器使用远程BigQuery服务进行身份验证？更好的是，有一个本地的BigQuery吗？

据我所知，AppAssertionCredentials目前不在本地开发服务器上工作，但它本身是非常令人沮丧的。 / p>

本地开发服务器沙箱以外的标准python代码的另一种方法详细介绍 在本地开发服务器上不起作用，因为即使启用了PyCrypto，沙箱也不允许使用某些posix模块，例如'pwd'。

我在远程服务器上使用 AppAssertionCredentials ，在本地python中使用 SignedJwtAssertionCredentials 本地，所以服务帐户设置正确。

在try / except块内的oauth2client / crypt.py内导入失败 - 在将它们注释掉沙箱白名单例外很容易看到。

我在向白名单中添加pwd，然后又出现了另一个问题，于是我急忙跑出那个兔子洞。 / p>

我试着将PyCrypto直接加入项目中，结果类似。

我也尝试过使用OpenSSL结果类似。

我找了一个本地特定于引擎的PyCrypto无济于事，我错过了一个？我应该说这是在Mac OSX上 - 也许我应该启动一个Linux的盒子，并给它一个去吗？

Google App Engine SDK的发布版在开发服务器上添加了对AppAssertionCredentials方法的支持。要在本地使用此方法，请将以下参数添加到 dev_appserver.py ：

  $ dev_appserver.py  - 帮助
 ... 
应用程序标识：
 --appidentity_email_address APPIDENTITY_EMAIL_ADDRESS 
与服务帐户关联的电子邮件地址，
可下载键。没有本地
应用程序标识可能为None。 （默认值：无）
 --appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH 
与服务
账户（.pem格式）关联的私钥文件的路径。如果
 appidentity_email_address已设置，则必须设置。 （默认：无）
  

使用这些：

1. 在 Google开发者控制台中，选择一个项目，然后导航至API和放大器;auth - >证书 - >创建新的客户端ID。

PKCS12（.p12）格式。请注意该服务帐户的电子邮件地址。 确保你将该服务帐户电子邮件地址添加到任何包含数据的项目的权限它需要访问，默认情况下它被添加到它创建的项目组中。 使用以下命令将PKCS12格式转换为PKCS1格式：




$ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass：notasecret | openssl rsa> /path/to/secret.pem




2. 开始 dev_appserver.py as：
   
   

   $ dev_appserver.py --appidentity_email_address [e-mail] --appidentity_private_key_path /path/to/secret.pem .. 。




3. 使用 appidentity 模块和 AppAssertionCredentials




请确保 /path/to/secret.pem 不在您的应用程序源目录中，因此它不会意外地部署为应用程序的一部分。

This is specifically a question relating to server to server authorisation between a python Google AppEngine app and Google's BigQuery, but could be relevant for other cloud services.

tldr; Is it possible to get the App Engine local development server to authenticate with the remote BigQuery service? Better yet is there a local BigQuery?

I understand that AppAssertionCredentials does not currently work on the local development server, though that in itself is very frustrating.

The alternative method which works for standard python code, outside of the local development server sandbox, detailed here does not work for the local development server because even with PyCrypto enabled the sandbox does not allow some posix modules e.g. 'pwd'.

I have got AppAssertionCredentials working on the remote server and the SignedJwtAssertionCredentials method working in native python locally, so the service accounts are set up properly.

The imports fail within oauth2client/crypt.py within the try/except blocks - after commenting them out the sandbox whitelist exceptions are easily seen.

I've fiddled around with adding 'pwd' to the whitelist, then another problem crops up, so I scurried back out of that rabbit hole.

I've tried including PyCrypto directly into the project with similar results.

I've also tried with OpenSSL with similar results.

I have looked for a local appengine specific PyCrypto to no avail, have I missed one? I should say this is on Mac OSX - perhaps I should fire up a linux box and give that a go?

解决方案

A recent release of Google App Engine SDK added support for the AppAssertionCredentials method on the development server. To use this method locally, add the following arguments to dev_appserver.py:

$ dev_appserver.py --help
...
Application Identity:
  --appidentity_email_address APPIDENTITY_EMAIL_ADDRESS
                        email address associated with a service account that
                        has a downloadable key. May be None for no local
                        application identity. (default: None)
  --appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH
                        path to private key file associated with service
                        account (.pem format). Must be set if
                        appidentity_email_address is set. (default: None)

To use these:

1. In Google Developer Console, select a project then navigate to "API & auth" -> "Credentials" -> "Create new client ID".

2. Select "Service account" and follow the prompts to download the private key in PKCS12 (.p12) format. Take note of the email address for the service account.

3. Make sure you add that service account email address to the "Permissions" tab for any project that contains data it needs to access, by default it is added to the project team in which it was created.

4. Convert the PKCS12 format to PKCS1 format using the following command:

   $ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem

5. Start dev_appserver.py as:

   $ dev_appserver.py --appidentity_email_address [e-mail] --appidentity_private_key_path /path/to/secret.pem ...

6. Use appidentity module and AppAssertionCredentials in the same manner locally as you normally would in production.

Please ensure that /path/to/secret.pem is outside of your application source directory so that it is not accidentally deployed as part of your application.

这篇关于无法从本地App Engine开发服务器访问BigQuery的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！