BigQuery表数据导出 [英] BigQuery Table Data Export

问题描述

我正在尝试使用python API从BigQuery表中导出数据.表包含1到4百万行.因此，我将maxResults参数设置为最大值，即100000，然后进行分页.但是问题是，在一页中，我只能得到2652行，因此分页数太多.任何人都可以提供原因或解决方案.格式为JSON. 还是可以不使用GCS将数据导出为CSV格式?

I am trying to export data from BigQuery Table using python api. Table contains 1 to 4 million of rows. So I have kept maxResults parameter to maximum i.e. 100000 and then paging through. But problem is that in One page I am getting 2652 rows only so number of paging is too much. Can anyone provide reason for this or solution to deal. Format is JSON. Or can I export data into CSV format without using GCS?

我尝试通过插入作业并保留allowLargeResults =true，但结果保持不变.

I tried by inserting job and keeping allowLargeResults =true, but the result remain same.

下面是我的查询正文:

queryData = {'query':query,
                     'maxResults':100000,
                     'timeoutMs':'130000'}

谢谢.

推荐答案

您可以尝试使用bq命令行工具

You can try to export data from table without using GCS by using bq command line tool https://cloud.google.com/bigquery/bq-command-line-tool like this:

bq --format=prettyjson query --n=10000000 "SELECT * from publicdata:samples.shakespeare"

您也可以根据需要使用--format=json.

You can use --format=json depending on your needs as well.

这篇关于BigQuery表数据导出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！