通过Windows服务在BigQuery表中插入记录 [英] Inserting a record in BigQuery table via windows service

问题描述

我创建了一个使用服务帐户授权与Google BigQuery API完美通信的Windows服务。为了测试目的，我可以从BigQuery表中提取数据，以检查我的Windows服务和BigQuery API之间的通信。但是这个服务将基本上从MSSQL服务器获取记录，然后逐一插入或附加记录到BigQuery中的目标表。我不想使用CSV或JSON文件作为数据源。有没有办法这样做？

I've created a windows service that communicates with the Google BigQuery API perfectly using Service Account authorization. For testing purposes, I'm able to fetch data from a BigQuery table to check communication between my Windows service and the BigQuery API. But this service will basically fetch records from MSSQL server and then one by one will insert or append a record to the target table in BigQuery. I don't want to use CSV or JSON files as a data source. Is there any way of doing so?

推荐答案

不，你必须使用CSV或JSON作为源格式

No, you'll have to use either CSV or JSON as a source format to load your data into BigQuery.

BigQuery目前支持批量摄取模型，而不支持记录级别模型。

BigQuery currently supports a batch ingestion model, not a record level model. The daily import quota is 1,000 import jobs per table and 10,000 import jobs total for the whole project.

我建议您每隔一段时间将数据导出MSSQL服务器（也许每小时），将输出缓存在CSV或JSON文件中，然后在MSSQL导出步骤完成后启动BigQuery提取作业。

I would suggest that you export data out of MSSQL server in intervals (perhaps hourly), cache the output in a CSV or JSON file, and then kick off a BigQuery ingestion job after the MSSQL export step is complete.

这篇关于通过Windows服务在BigQuery表中插入记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！