使用云功能执行数据流模板时出错 [英] Error while executing Dataflow template using cloud function

问题描述

尝试使用Google Cloud函数执行自定义数据流模板时遇到错误.

Getting below error while trying to execute custom dataflow template using Google Cloud function.

错误:问题正在运行的数据流模板，错误是:{错误:尝试检索Compute Engine内置服务帐户的访问令牌时，返回了禁止的错误.这可能是因为Compute Engine实例没有指定了正确的权限范围.无法刷新访问令牌".

Error:"problem running dataflow template, error was: { Error: A Forbidden error was returned while attempting to retrieve an access token for the Compute Engine built-in service account. This may be because the Compute Engine instance does not have the correct permission scopes specified. Could not refresh access token".

我尝试提供所有必需的权限和范围.有人可以提出解决方案.

I have tried supplying all the required permissions and scopes.Could someone please suggest a resolution.

推荐答案

google-cloud 节点库尚不支持Dataflow API，因此当前使用该API的方法是

The google-cloud node library does not yet support the Dataflow API, so the current way to use that API is the googleapis library.

按照那里的说明，我尝试使用

Following the instructions there, I've tried to launch a Dataflow job with a Google-provided template using an HTTP-triggered function, and had no issues:

const {google} = require('googleapis');
const project = "your-project-id"

exports.launchDataflowTemplate = (req, res) => {
    let result;
    google.auth.getApplicationDefault(function(err, authClient, projectId) {
            if (err) {
                throw err;
            }
            if (authClient.createScopedRequired && authClient.createScopedRequired()) {
                authClient = authClient.createScoped([
                    'https://www.googleapis.com/auth/cloud-platform',
                    'https://www.googleapis.com/auth/compute',
                    'https://www.googleapis.com/auth/compute.readonly',
                    'https://www.googleapis.com/auth/userinfo.email'
                ]);
            }
            var dataflow = google.dataflow({
                version: "v1b3",
                auth: authClient
            });

            var launchParams = {
                "inputFilePattern": "gs://your-input-bucket/*.gz",
                "outputDirectory": "gs://your-result-bucket/",
                "outputFailureFile": "gs://your-logs-bucket/error.csv"
            };

            var env = {
               "tempLocation": "gs://your-staging-bucket/temp",
               "zone": "us-central1-f"
            }


            var opts = {
                projectId: project,
                gcsPath: "gs://dataflow-templates/latest/Bulk_Decompress_GCS_Files",
                resource: {
                    parameters: launchParams,
                    environment: env
                }
            };

            dataflow.projects.templates.launch(opts, (err, result) => {
                if (err) {
                    throw err;
                }
                res.send(result.data);
            });
    });
};

这篇关于使用云功能执行数据流模板时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！