CLI“ bq load” -如何使用不可打印字符作为分隔符? [英] CLI "bq load" - how to use non-printable character as delimiter?
问题描述
我无法将数据作为单列行加载到BigQuery中。我希望BigQuery能够提供无定界符作为选项,但是与此同时,我需要选择我能找到的最模糊的ASCII定界符,这样我的单列行就不会分成几列。
I'm having trouble loading data into BigQuery as a single column row. I wish BigQuery offered the ability to have "no delimiter" as an option, but in the meantime I need to choose the most obscure ASCII delimiter I can find so my single column row is not split into columns.
执行此操作时,CLI不允许我输入奇怪的字符,因此我需要通过Python或其他渠道使用API。
When doing this the CLI won't allow me to input strange characters, so I need to use the API through Python or other channels.
如何使用带有不可打印字符的CLI代替?
How can I use the CLI instead with a non printable character?
来自 BigQuery惰性数据加载:DDL,DML,分区和半万亿维基百科网页浏览量:
#!/bin/python
from google.cloud import bigquery
bq_client = bigquery.Client(project='fh-bigquery')
table_ref = bq_client.dataset('views').table('wikipedia_views_gcs')
table = bigquery.Table(table_ref, schema=SCHEMA)
extconfig = bigquery.ExternalConfig('CSV')
extconfig.schema = [bigquery.SchemaField('line', 'STRING')]
extconfig.options.field_delimiter = u'\u00ff'
extconfig.options.quote_character = ''
推荐答案
To使用带有BQ加载的不可打印字符,可以在bash中使用 echo
:
To use a non-printable character with BQ load you can use echo
in bash:
bq load \
--source_format=CSV \
--field_delimiter=$(echo -en "\x01") \
--noreplace --max_bad_records=100 \
<bq_dataset>.<bq_table> gs://<bucket_name>/<file_name>.csv
这篇关于CLI“ bq load” -如何使用不可打印字符作为分隔符?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!