加载操作中的BigQuery错误:叶字段总数过多 [英] BigQuery error in load operation: Too many total leaf fields

问题描述

我正在将一个大文件上传到BigQuery中，因为它太慢了，无法在我自己的PC上运行.bq --location = EU负载--field_delimiter ='\ t'--skip_leading_rows = 1 --source_format = CSV single_cells.retinal_bipolar gs://single_cells/retinal-bipolar-neuron-drop-seq/exp_matrix.txt ./schema.json但是，我遇到了一个错误:

I am uploading a big file into BigQuery, because it is too slow to operate on my own PC. bq --location=EU load --field_delimiter='\t' --skip_leading_rows=1 --source_format=CSV single_cells.retinal_bipolar gs://single_cells/retinal-bipolar-neuron-drop-seq/exp_matrix.txt ./schema.json However, I got an error:

加载操作中的BigQuery错误:叶字段总数过多:27500

BigQuery error in load operation: Too many total leaf fields: 27500

实际上，数据有27500列；BigQuery不允许这样做吗?

Indeed, the data has 27500 columns; doesn't BigQuery allow this?

推荐答案

另一种解决方法-正如Elliott指出的那样-您可以将文件加载到BigQuery表中，该表设置的架构只有一个类型为STRING的列.然后(假设文件和行的大小不超过 大小限制 )-您将能够使用BigQuery的所有功能！

Another workaround - as Elliott pointed - you can load your file into BigQuery table that is set with schema having ONLY one column of type STRING. Then (assuming that file and row size does not exceed size limitations) - you will be able to use all power of BigQuery!

这篇关于加载操作中的BigQuery错误:叶字段总数过多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！