如何从云功能中读取.json类型的新云存储文件的内容？ [英] How do I read the contents of a new cloud storage file of type .json from within a cloud function?

问题描述

传递给我的Google云计算功能的事件只会告诉我存储桶和文件的名称，以及文件是否已删除。是的，那里还有更多，但似乎没有那么有用：

The event passed to my Google cloud function only really tells me the name of the bucket and file, and whether the file was deleted. Yes, there's more there, but it doesn't seem all that useful:

{ timestamp: '2017-03-25T07:13:40.293Z', 
eventType: 'providers/cloud.storage/eventTypes/object.change', 
resource: 'projects/_/buckets/my-echo-bucket/objects/base.json#1490426020293545', 
data: { kind: 'storage#object', 
       resourceState: 'exists', 
       id: 'my-echo-bucket/base.json/1490426020293545', 
       selfLink: 'https://www.googleapis.com/storage/v1/b/my-echo-bucket/o/base.json', 
       name: 'base.json', 
       bucket: 'my-echo-bucket', 
       generation: '1490426020293545', 
       metageneration: '1', 
       contentType: 'application/json', 
       timeCreated: '2017-03-25T07:13:40.185Z', 
       updated: '2017-03-25T07:13:40.185Z', 
       storageClass: 'STANDARD', 
       size: '548', 
       md5Hash: 'YzE3ZjUyZjlkNDU5YWZiNDg2NWI0YTEyZWZhYzQyZjY=', 
       mediaLink: 'https://www.googleapis.com/storage/v1/b/my-echo-bucket/o/base.json?generation=1490426020293545&alt=media', contentLanguage: 'en', crc32c: 'BQDL9w==' } 
}

我如何获得内容而不仅仅是上传到gs存储桶的新.json文件的元数据？

How do I get the contents and not merely the meta-data of a new .json file uploaded to a gs bucket?

我尝试使用 npm：request（） on event.data.selfLink ，这是存储桶中文件的URL，然后返回授权错误：

I tried using npm:request() on event.data.selfLink, which is a URL for the file in the storage bucket, and got back an authorization error:

code：401，message：匿名用户没有storage.objects.get访问对象my- echo-bucket / base.json。

关于读取存储桶的问题也有类似的问题，但可能在不同的平台上。无论如何，未答复：

There was a similar question on SO about reading storage buckets, but probably on a different platform. Anyway it was unanswered:

如何使用javascript在Google云端存储上读取文件内容
`

推荐答案

您需要使用客户端库进行谷歌存储，而不是通过URL访问。谷歌使用 request（）只有在文件公开访问时才有效。

You need to use a client library for google storage instead of accessing via the URL. Using request() against the URL would only work if the file was exposed to public access.

导入谷歌包含项目的npm管理目录中的云存储库。

Import the google cloud storage library in the npm-managed directory containing your project.

npm i @google-cloud/storage -S

google-cloud / storage的npm页面有不错的例子，但我不得不仔细阅读API，看看下载到内存的简单方法。

The npm page for google-cloud/storage has decent examples but I had to read through the API a bit to see an easy way to download to memory.

在Google云端功能环境中，您无需向初始化存储提供任何API密钥等。

Within the Google Cloud Functions environment, you do not need to supply any api key, etc. to storage as initialization.

const storage = require('@google-cloud/storage')();

传递的有关文件的元数据可用于确定您是否真的想要该文件。

The metadata passed about the file can be used to determine if you really want the file or not.

如果需要该文件，可以使用 file.download 函数，它可以采用回调函数，或者缺少回调函数，将返回一个promise。 >
然而，数据以缓冲区的形式返回，因此您需要调用 data.toString（'utf-8'）将其转换为utf-8编码的字符串。

When you want the file, you can download it with the file.download function, which can take either a callback or, lacking a callback, will return a promise.
The data however, is returned as a Buffer so you will need to call data.toString('utf-8') to convert it to a utf-8 encoded string.

const storage = require('@google-cloud/storage')();

exports.logNewJSONFiles = function logNewJSONFiles(event){
    return new Promise(function(resolve, reject){
        const file = event.data;
        if (!file){
            console.log("not a file event");
            return resolve();
        }
        if (file.resourceState === 'not_exists'){
            console.log("file deletion event");
            return resolve();
        }
        if (file.contentType !== 'application/json'){
            console.log("not a json file");
            return resolve();
        }
        if (!file.bucket){
            console.log("bucket not provided");
            return resolve();
        }
        if (!file.name){
            console.log("file name not provided");
            return resolve();
        }
        (storage
         .bucket(file.bucket)
         .file(file.name)
         .download()
         .then(function(data){
             if (data)
                 return data.toString('utf-8');
         })
         .then(function(data){
             if (data) {
                 console.log("new file "+file.name);
                 console.log(data);
                 resolve(data);
             }
         })
         .catch(function(e){ reject(e); })
             );
    });
};

部署符合预期：

gcloud beta functions deploy logNewJSONFiles --stage-bucket gs://my-stage-bucket --trigger-bucket gs://my-echo-bucket

请记住在Google Cloud Platform上的Stackdriver：Logging页面查看 console.log 条目。

Remember to look in the Stackdriver:Logging page on Google Cloud Platform for the console.log entries.

更新：（2017年3月28日）。上面的代码天真地假设传输在第一次尝试时完成。目前，当尝试从Google Cloud Functions使用Google存储时，会看到相当多的 ECONNRESET 查杀。希望这会有所改善，但与此同时......使用 npm：promise-retry 有助于通常，在 ECONNRESET 之后，下次尝试时转移会通过OK。 promise-retry默认会尝试多达10次。

UPDATE: (Mar 28, 2017). The code above naively assumes that transfers complete OK on the first attempt. Currently seeing quite a few ECONNRESET killing transfers when trying to use Google Storage from Google Cloud Functions. Hopefully this improves, but in the meantime... using npm:promise-retry helps as generally the transfer goes through OK on the next attempt after ECONNRESET. promise-retry will try up to 10 times by default.

上面代码的最新承诺重试版现在位于 npm：maybe-json 。为了写作，我将 npm：pipe-to-storage 放在一起使用如果字符串或返回新的可读流的函数用作第一个参数，则promise-retry。

The latest promise-retry version of the code above now lives at npm:maybe-json. For writing I've thrown together npm:pipe-to-storage which will use promise-retry if a string or a function returning a new readable stream is used as the first parameter.

这篇关于如何从云功能中读取.json类型的新云存储文件的内容？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！