自动将文件从 Google Cloud Storage 上传到 Bigquery [英] Automate file upload from Google Cloud Storage to Bigquery

问题描述

我们设置了自动 FTP 流程，每天将数据文件导入 Google Cloud Storage.

We have an automated FTP process set up which imports a data file into Google Cloud Storage daily.

我想设置一个每日自动化作业，将这个 csv 上传到一个 bigquery 表中.

I would like to set up a daily automated job that uploads this csv into a bigquery table.

这样做的最佳方法是什么?我目前的第一个想法是使用每天运行 python 脚本的 cron 作业设置一个应用引擎实例.有没有更好的解决方案?

What is the best way to do this? My current first thought is to set up an app engine instance with a cron job that runs a python script every day. Is there a better solution?

推荐答案

带有云存储触发器<的后台云函数/code> 是您最好的选择！

您可以将其设置为监控新文件的特定存储桶并在触发触发器时执行加载脚本

You can set it to monitor specific bucket for new files and execute load script whenever trigger is fired

忘记提及 - Cloud Functions 支持(截至目前)仅用于脚本编写的 node.js - 这通常不是问题，但只是想提及 :o)

Forgot to mention - Cloud Functions support (as of now) only node.js for scripting - which usually not a problem but just wanted to mention :o)

这篇关于自动将文件从 Google Cloud Storage 上传到 Bigquery的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！