如何从bigquery提取csv并使用python将其发送到外部服务器? [英] How to extract a csv from bigquery and send it to an external server with python?

问题描述

我想自动将csv文件提取过程从Google BigQuery到Google Cloud Storage Bucket，再从后者到带有两个Python脚本的外部服务器，您能帮我吗?我会很感激的.

I would like to automate a csv file extraction process from Google BigQuery to a Google Cloud Storage Bucket, and from the latter to an external server with two Python scripts, could you help me please? I would appreciate it.

推荐答案

要从Python中的BigQuery中提取，您可以使用

For extracting from BigQuery in Python, you can use the Python Client for Google BigQuery.

以下基于此存储库的代码段应该可以帮助您

The below snippet based on this repository should get you going:

# client = bigquery.Client()
# bucket_name = 'my-bucket'
project = "bigquery-public-data"
dataset_id = "samples"
table_id = "shakespeare"

destination_uri = "gs://{}/{}".format(bucket_name, "shakespeare.csv")
dataset_ref = bigquery.DatasetReference(project, dataset_id)
table_ref = dataset_ref.table(table_id)

extract_job = client.extract_table(
    table_ref,
    destination_uri,
    # Location must match that of the source table.
    location="US",
)  # API request
extract_job.result()  # Waits for job to complete.

print(
    "Exported {}:{}.{} to {}".format(project, dataset_id, table_id, destination_uri)
)

为了将导出发布到另一台服务器，您可以使用云用于Python的存储客户端库，用于将CSV文件发布到您选择的服务器或服务中.

In order to post the export to another server, you can use the Cloud Storage Client Library for Python to post the CSV file to your server or service of choice.

这篇关于如何从bigquery提取csv并使用python将其发送到外部服务器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！