将CSV导入BigQuery中的表格时无法添加字段 [英] Cannot add fields when importing CSV to a table in BigQuery
本文介绍了将CSV导入BigQuery中的表格时无法添加字段的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我有一个python脚本执行一个gbq作业,将一个csv文件f加载到BigQuery中的表格中。我尝试以csv格式上传数据并出现以下错误:
I have a python script that executes a gbq job to load a csv file f to table in BigQuery. I am trying to upload data in csv format and getting the following error:
400 Invalid schema update. Cannot add fields (field: string_field_8)
这是我的csv:
id,first_name,username,last_name,chat_username,chat_id,forward_date,message_text
231125223|Just|koso|swissborg_bounty|-1001368946079|1517903147|tes
481895079|Emerson|EmersonEmory|swissborg_bounty|-1001368946079|1517904387|pictu
316560356|Ken Sam|ICOnomix|swissborg_bounty|-1001368946079|1517904515|Today
这是我的代码:
from google.cloud.bigquery import Client
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '***.json'
os.environ['GOOGLE_CLOUD_DISABLE_GRPC'] = 'True'
from google.cloud import bigquery
dataset_name = 'test_temporary_dataset'
table_name='table_telega'
bigquery_client = bigquery.Client()
dataset = bigquery_client.dataset(dataset_name)
table = dataset.table(table_name)
job_config = bigquery.LoadJobConfig()
job_config.source_format = 'text/csv'
job_config.skip_leading_rows = 1
job_config.autodetect = True
job_config.fieldDelimiter='|'
job_config.allow_jagged_rows=True
job_config.ignoreUnknownValues=True
job_config.allow_quoted_newlines=True
with open('**.csv', 'rb') as source_file:
#job = table.upload_from_file(source_file, source_format='text/csv')
job=bigquery_client.load_table_from_file(source_file, table, job_config=job_config)
job.result()
print(job.result())
如何解决它?我应该改变什么?
how to fix it? what should I change ?
推荐答案
只需在代码中添加此行
job_config._properties['load']['schemaUpdateOptions'] = ['ALLOW_FIELD_ADDITION']
,这将允许将列添加到您现有的模式中。
and this will allow column addition to your existing schema.
这篇关于将CSV导入BigQuery中的表格时无法添加字段的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文