如何使用python将本地CSV上传到Google大查询 [英] How to upload a local CSV to google big query using python
本文介绍了如何使用python将本地CSV上传到Google大查询的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在尝试使用python将本地CSV上传到Google大查询
I'm trying to upload a local CSV to google big query using python
def uploadCsvToGbq(self,table_name):
load_config = {
'destinationTable': {
'projectId': self.project_id,
'datasetId': self.dataset_id,
'tableId': table_name
}
}
load_config['schema'] = {
'fields': [
{'name':'full_name', 'type':'STRING'},
{'name':'age', 'type':'INTEGER'},
]
}
load_config['sourceFormat'] = 'CSV'
upload = MediaFileUpload('sample.csv',
mimetype='application/octet-stream',
# This enables resumable uploads.
resumable=True)
start = time.time()
job_id = 'job_%d' % start
# Create the job.
result = bigquery.jobs.insert(
projectId=self.project_id,
body={
'jobReference': {
'jobId': job_id
},
'configuration': {
'load': load_config
}
},
media_body=upload).execute()
return result
当我运行此命令时,它会引发类似
when I run this it throws error like
"NameError:未定义全局名称'MediaFileUpload'"
"NameError: global name 'MediaFileUpload' is not defined"
是否需要任何模块,请帮助.
whether any module is needed please help.
推荐答案
pip install --upgrade google-api-python-client
然后在您的python文件顶部写入:
Then on top of your python file write:
from googleapiclient.http import MediaFileUpload
但是请注意,您会错过一些括号.更好地写:
But care you miss some parenthesis. Better write:
result = bigquery.jobs().insert(projectId=PROJECT_ID, body={'jobReference': {'jobId': job_id},'configuration': {'load': load_config}}, media_body=upload).execute(num_retries=5)
顺便说一句,您将上载所有CSV行,包括定义列的第一行.
And by the way, you are going to upload all your CSV rows, including the top one that defines columns.
这篇关于如何使用python将本地CSV上传到Google大查询的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文