将位于Google云存储中的文本文件(.txt)转换为CSV文件? [英] convert text files(.txt) located in google cloud storage into CSV files?

问题描述

我在云端存储中有一组文本(.txt)文件(每5分钟上传到云端存储中一次).我想做的是将它们上传到大型查询中.但是bIgquery无法接受文本文件.因此，我必须将其转换为Bq可接受的格式.最好的方法是什么?

I have a set of text(.txt) files in cloud storage(that are uploaded into the Cloud storage in every 5 minutes). What I want to do is I want to upload them into big query. But bIgquery cant accept text files. So I have to convert it to Bq acceptable format. What is the best possible way?

推荐答案

按照本文档，BigQuery仅支持加载以下文件格式的数据:CSV，JSON，Avro和Google Cloud Datastore备份.

As per this document, BigQuery only supports loading data with the following file format: CSV, JSON, Avro and Google Cloud Datastore backups.

因此，如果您将文本文件上传到BigQuery，则BigQuery会将文本文件读取为CSV文件，然后确实会出错.

Hence, if you upload a Text file to BigQuery, BigQuery reads your text file as a CSV file and then really would run into an error.

在将文本文件上传到BigQuery之前，您必须将其手动转换为CSV文件.

You would have to manually convert your text file into a CSV file, before uploading it to BigQuery.

或者，您也可以使用Cloud Dataprep，因为它支持文本文件作为输入.您可以在Dataprep中使用文本文件进行转换，然后将结果导出到BigQuery.

Alternatively, you may also use Cloud Dataprep as it supports text files as inputs. You may do transformations with your text file here in Dataprep then export the results to BigQuery.

这是 Dataprep概述和

Here is an Overview of Dataprep and a Quickstart Documentation to learn how to use it.

这篇关于将位于Google云存储中的文本文件(.txt)转换为CSV文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！