R到BigQuery数据上传错误 [英] R to BigQuery Data Upload Error

问题描述

我正在使用R包"bigrquery"将R数据框中的数据上传到现有的BigQuery表中，如下所示:

I'm using the R package "bigrquery" to upload data from an R data frame into an existing BigQuery table as follows:

mybq = bq_table(project='...', dataset='...', table=...)
bq_table_upload(x=mybq, values=..., create_disposition='CREATE_NEVER', 
                write_disposition='WRITE_APPEND')

但是我收到以下错误消息:

But I'm the following error message:

错误:无效的架构更新.字段NewID的类型已从"STRING"更改为"INTEGER"

Error: Invalid schema update. Field NewID has changed type from STRING to INTEGER

BigQuery似乎正在自动检测数据格式，并且错误地认为NewID列(其值为"00487")实际上是字符串时为数字.当我在NewID值后附加一个"x"字符时，该错误就会消失，并且上传功能会正常运行.使用"bigrquery"包上传数据时，有什么方法可以禁用自动检测?

BigQuery appears to be auto-detecting the data format and mistakenly thinks the NewID column, which has values like "00487", is numeric when it's actually a string. When I append an "x" character to the NewID values, then the error goes away and the upload functions perfectly. Is there any way to disable the auto-detection when uploading data using the "bigrquery" package?

推荐答案

同一库中的bq_perform_load函数应该是对此的解决方案.在此函数中，您可以使用参数fields指定架构，以便Bigquery不会按照

The bq_perform_load function in the same library should be a solution for this. In this function you can specify the schema with parameter fields so Bigquery will not auto-detect schema as explained here.

我进行了测试，并且最终成功了.我创建了一个包含两列(STRING，STRING)的表，这是我的源数据:

I tested and it worked on my end. I created a table with two columns (STRING, STRING) and this is my source data:

0017    0015
0123    1234
1022    1202

我在R中运行以下命令以使其工作:

I run the following command in R to make it work:

bq_perform_load('MY_PROJECT.MYDATASET.MYTABLE', "MY_GCS_OBJECT_LINK", nskip = 0, fields = list(bq_field("test1", "string"),bq_field("test2", "string")) , source_format = "CSV",create_disposition = "CREATE_NEVER", write_disposition = "WRITE_APPEND")

注意::我第一次尝试用fields = NULL运行相同的命令，但失败了.

Note: I tried it at first time to run the same command with fields = NULL and it failed.

这篇关于R到BigQuery数据上传错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！