如何在应用程序引擎和应用程序中使用Bigquery streaming insertall蟒蛇 [英] How to use Bigquery streaming insertall on app engine & python

问题描述

根据Google的文档，有一种简单的方法可以将数据流式传输到bigquery ：

• http://googlecloudplatform.blogspot.co.il/2013/09/google-bigquery -goes-real-time-with-streaming-inserts-time-based-queries-and-more.html

https://developers.google.com/bigquery/streaming-data-into-bigquery#streaminginsertexamples
（注意：在上面的链接中，您应该选择python选项卡而不是Java）


$ b

以下是关于流式插入应如何编码的示例代码片段：

  body = {rows：[
 {json：{column_name：7.7，}} 
]} 
 
 response = bigquery.tabledata（）。insertAll（
 projectId = PROJECT_ID，
 datasetId = DATASET_ID，
 tableId = TABLE_ID，
 body = body）.execute（）
  

虽然我已经下载了客户端api，但是我没有发现任何引用上述Google示例中引用的bigquery模块/对象。





bigquery对象（来自代码段）应该位于哪里？





任何人都可以使用这个代码片段与正确的进口）？





我一直在寻找那么多，发现文档混淆和局部。

解决方案

  import httplib2 
最小工作量（只要您填写项目的正确ID） 
从apiclient导入发现
从oauth2client导入appengine 
 
 _SCOPE ='https://www.googleapis.com/auth/bigquery'
 
＃更改以下3个值：
 PROJECT_ID ='your_project'
 DATASET_ID ='your_dataset'
 TABLE_ID ='TestTable'
 
 
 body = {rows ：[
 {json：{Col1：7，}} 
]} 
 
 credentials = appengine.AppAssertionCredentials（scope = _SCOPE）
 http = credentials.authorize（httplib2.Http（））
 
 bigquery = discovery.build（'bigquery'，'v2'，http = http）
 response = bigquery.tabledata（）。insertAll （
 projectId = PROJECT_ID，
 datasetId = DATASET_ID，
 tableId = TABLE_ID，
 body = body）.ex ecute（）
 
打印回复
  

正如乔丹所说：请注意，这使用appengine机器人对BigQuery进行身份验证，因此您需要将机械手帐户添加到数据集的ACL中。请注意，如果您还想使用机器人运行查询，而不仅仅是流式处理，则需要机器人成为项目团队的成员，以便它有权运行作业。




I would like to develop an app engine application that directly stream data into a BigQuery table.

According to Google's documentation there is a simple way to stream data into bigquery:

• http://googlecloudplatform.blogspot.co.il/2013/09/google-bigquery-goes-real-time-with-streaming-inserts-time-based-queries-and-more.html

• https://developers.google.com/bigquery/streaming-data-into-bigquery#streaminginsertexamples (note: in the above link you should select the python tab and not Java)

Here is the sample code snippet on how streaming insert should be coded:

body = {"rows":[
{"json": {"column_name":7.7,}}
]}

response = bigquery.tabledata().insertAll(
   projectId=PROJECT_ID,
   datasetId=DATASET_ID,
   tableId=TABLE_ID,
   body=body).execute()

Although I've downloaded the client api I didn't find any reference to a "bigquery" module/object referenced in the above Google's example.

Where is the the bigquery object (from snippet) should be located?

Can anyone show a more complete way to use this snippet (with the right imports)?

I've Been searching for that a lot and found documentation confusing and partial.

解决方案

Minimal working (as long as you fill in the right ids for your project) example:

import httplib2
from apiclient import discovery
from oauth2client import appengine

_SCOPE = 'https://www.googleapis.com/auth/bigquery'

# Change the following 3 values:
PROJECT_ID = 'your_project'
DATASET_ID = 'your_dataset'
TABLE_ID = 'TestTable'


body = {"rows":[
    {"json": {"Col1":7,}}
]}

credentials = appengine.AppAssertionCredentials(scope=_SCOPE)
http = credentials.authorize(httplib2.Http())

bigquery = discovery.build('bigquery', 'v2', http=http)
response = bigquery.tabledata().insertAll(
   projectId=PROJECT_ID,
   datasetId=DATASET_ID,
   tableId=TABLE_ID,
   body=body).execute()

print response

As Jordan says: "Note that this uses the appengine robot to authenticate with BigQuery, so you'll to add the robot account to the ACL of the dataset. Note that if you also want to use the robot to run queries, not just stream, you need the robot to be a member of the project 'team' so that it is authorized to run jobs."

这篇关于如何在应用程序引擎和应用程序中使用Bigquery streaming insertall蟒蛇的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！