从 Google BigQuery 导出到 CloudSQL? [英] Export from Google BigQuery into CloudSQL?

问题描述

我想将 BigQuery 中的数据(即报告)转储到 CloudSQL 数据库中，以编程方式实现此目的的最佳方法是什么?

I want to dump data from BigQuery (i.e. reports) into a CloudSQL database, what is the best way to achieve this programatically?

我意识到我可以手动通过运行BigQuery 查询，将其下载为 CSV，然后通过云控制台上传，但我想以编程方式执行此操作，最好使用 Python/SQL.

I realise I could do this manually by running a BigQuery query, downloading it as a CSV, then uploading it through the Cloud console, but I want to do this programatically, preferably in Python/SQL.

推荐答案

如果您想转储整个表，可以结合使用 BigQuery 和 Cloud SQL API 来实现这一点.

If you would like to dump entire tables, you can use a combination of the BigQuery and Cloud SQL APIs to achieve this.

BigQuery 文档在 python 中有一个 API 示例将 BigQuery 表提取到 Cloud Storage.

The BigQuery documentation has an API example in python for extracting a BigQuery table to Cloud Storage.

一旦数据在云存储中，您就可以使用用于将数据导入 MySQL 表的 Cloud SQL Admin API.

Once the data is in Cloud Storage, you can use the Cloud SQL Admin API to import the data into a MySQL table.

如果您需要更精细的控制，您可以使用 BigQuery API 来执行查询、获取结果、连接到 Cloud SQL 实例并插入数据.如果数据量很大，这将不会表现得很好.

If you need more granular control, you can use the BigQuery API to perform the query, fetch the results, connect to the Cloud SQL instance and insert the data. This won't perform as well if the amount of data is large.

更复杂的方法是使用 Dataflow 将您感兴趣的数据写入 Cloud Storage，然后使用 Cloud SQL API 将其导入.

A more complex approach is to use Dataflow to write the data you are interested in to Cloud Storage and use the Cloud SQL API to import it.

(出于我自己的好奇，您能否描述一下需要 Cloud SQL 而不是 BigQuery 中的数据的用例?这将帮助我/我们了解我们的客户如何使用我们的产品以及我们可以改进的地方.)

(For my own curiosity, can you describe the use case for wanting the data in Cloud SQL instead of BigQuery? It will help me/us understand how our customers are using our product and where we can improve.)

这篇关于从 Google BigQuery 导出到 CloudSQL?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！