将查询结果从BigQuery导出到Postgres [英] Export query results from BigQuery to Postgres

问题描述

我正在尝试在BigQuery中导出查询结果，并将数据获取到Postgres中.数据可能多达2.5亿条记录，约26Gb.

I am trying to export the results of a query in BigQuery, and get the data into Postgres. The data may be as much 250 million record, ~26Gb.

选项1:

将查询结果保存到临时表中将表格导出到CSV批量上传到Postgres

Save query results to a temp table Export table to csv(s) Bulk upsert to postgres

(这会很慢)

选项2:

以某种方式让两个DB直接讲话我不知道这是否可能

Somehow get the two DBs to speak directly I don't know if this is possible

谢谢您提供任何信息！

推荐答案

此用于PostgreSQL的BigQuery外部数据包装器可让您直接从PostgreSQL中查询BigQuery.使用此代码和 SELECT INTO 语句可以使您有效地并直接从BigQuery复制到PostgreSQL.

This BigQuery Foreign Data Wrapper for postgreSQL allows you to query BigQuery directly from within PostgreSQL. Using this and a SELECT INTO statement should allow you to efficiently and directly copy from BigQuery to PostgreSQL.

这篇关于将查询结果从BigQuery导出到Postgres的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！