在Cloud Build步骤中执行BigQuery查询 [英] Execute a BigQuery query in Cloud Build step

问题描述

我正在将Cloud Build与gcloud构建器一起使用.我将entrypoint覆盖为bq，因此可以在构建步骤中运行一些BigQuery SQL.以前，我将SQL直接嵌入到Cloud Build的YAML配置中.效果很好:

I'm using Cloud Build with the gcloud builder. I override the entrypoint to be bq so I can run some BigQuery SQL in my build step. Previously, I had the SQL embedded directly in the YAML config for Cloud Build. This works fine:

steps:
- name: gcr.io/cloud-builders/gcloud
  entrypoint: 'bq'
  args: ['query', '--use_legacy_sql=false', 'SELECT 1']

现在，我想将SQL从YAML重构到文件中.根据此处，您可以文件或将其通过管道传输到bq.这可以在命令行上正常使用.

Now I'd like to refactor the SQL out of the YAML and into a file instead. According to here, you can cat the file or pipe it to bq. This works on the command line without any problems.

但是，我无法使其与Cloud Build一起使用.我尝试了很多不同的组合，并转义了字符等.但是无论我尝试什么，shell都不会评估/执行cat my_query.sl反引号，而是认为它就是查询本身:

But, I can't get it to work with Cloud Build. I've tried lots of different combinations, and escaping chars etc. but no matter what I try the shell doesn't evaluate/execute the cat my_query.sl backticks, and instead thinks that it's the query itself:

工作正常:

在Cloud Build中构建无法使用:

steps:
- name: gcr.io/cloud-builders/gcloud
  entrypoint: 'bq'
  args: ['query', '--use_legacy_sql=false', '`cat my_query.sql`']

我也尝试使用管道传输而不是使用cat，但是出现了相同的错误.

I also tried piping it instead of using cat, but I get the same error.

我肯定在这里遗漏了一些明显的东西，但是我看不到.我可以构建一个自定义的docker映像，并将所有内容包装在shell脚本中，但我宁愿不必这样做(如果可能).

I must be missing something obvious here, but I can't see it. I could build a custom docker image, and wrap everything in a shell script, but I'd rather not have to do that if possible.

您如何在构建步骤中将Cloud Build与Shell评估一起使用?

How do you use Cloud Build with shell evaluation inside a build step?

推荐答案

您可以创建自定义的Bash脚本，例如:

You can create a custom Bash script, e.g.:

#!/bin/bash
if [ $# -eq 0 ]; then
  echo "No arguments supplied"
fi
bq query --use_legacy_sql=false < $1

将此名称命名为run_query.sh，然后将您的步骤定义为:

Name this run_query.sh, then define your steps as:

steps:
- name: gcr.io/cloud-builders/gcloud
  entrypoint: 'bash'
  args: ['run_query.sh', 'my_query.sql']

免责声明:这是基于阅读文档的内容，但是我实际上并没有使用Cloud Build.

Disclaimer: this is based on reading the docs, but I haven't actually used Cloud Build.

这篇关于在Cloud Build步骤中执行BigQuery查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！