从存储桶文件创建表的问题 [英] Issues creating table from bucket file

问题描述

我有一张很大的桌子（大约一千万行），我试图把它拉进我的大问题。由于创建表格时的大小限制，我不得不将CSV上传到存储桶中。当我尝试使用Datastore创建表时，作业失败，并显示错误消息：

 错误原因：无效。在故障排除错误中获取有关此错误的更多信息：无效。 
错误：
 gs：//es_main/provider.csv不包含有效的备份元数据。 
工作识别码：liquid-cumulus：job_KXxmLZI0Ulch5WmkIthqZ4boGgM 
开始时间：2015年12月16日下午3:00:51 
结束时间：2015年12月16日下午3:00:51 
目的地表：liquid-cumulus：ES_Main.providercloudtest 
源URI：gs：//es_main/provider.csv 
源格式：数据存储备份
  

通过使用同一个表中的小行样本文件并使用表创建中的CSV功能上传无任何错误，我遇到了麻烦，并且可以查看数据就好了。

我只是想知道元数据应该如何设置，以及存储桶中的编辑元数据选项，或者是否有其他解决方法。谢谢

解决方案

您发布的作业的错误消息告诉您，您提供的文件不是

I have a big table (About 10 million rows) that I'm trying to pull into my bigquery. I had to upload the CSV into the bucket due to the size constraints when creating the table. When I try to create the table using the Datastore, the job fails with the error:

Error Reason:invalid. Get more information about this error at Troubleshooting Errors: invalid.
Errors:
gs://es_main/provider.csv does not contain valid backup metadata.
Job ID: liquid-cumulus:job_KXxmLZI0Ulch5WmkIthqZ4boGgM
Start Time: Dec 16, 2015, 3:00:51 PM
End Time: Dec 16, 2015, 3:00:51 PM
Destination Table: liquid-cumulus:ES_Main.providercloudtest
Source URI: gs://es_main/provider.csv
Source Format: Datastore Backup

I've troubleshot by using a small sample file of rows from the same table and just uploading using the CSV feature in the table creation without any errors and can view the data just fine.

I'm just wondering what the metadata should be set as with the "Edit metadata" option within the bucket or if there is some other work around I'm missing. Thanks

解决方案

The error message for the job that you posted is telling you that the file you're providing is not a Datastore Backup file. Note that "Datastore" here means Google Cloud Datastore, which is another storage solution that it sounds like you aren't using. A Cloud Datastore Backup is a specific file type from that storage product which is different from CSV or JSON.

Setting the file metadata within the Google Cloud Storage browser, which is where the "Edit metadata" option you're talking about lives, should have no impact on how BigQuery imports your file. It might be important if you were doing something more involved with your file from Cloud Storage, but it isn't important to BigQuery as far as I know.

To upload a CSV file from Google Cloud Storage to BigQuery, make sure to select the CSV source format and the Google Storage load source as pictured below.

这篇关于从存储桶文件创建表的问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！