Cloud Function - 获取超过 10 MB 的文件内容 [英] Cloud Function - Getting file contents more than 10 MB

问题描述

Google Cloud Function 的配额页面:https://cloud.google.com/functions/quotas 表示 HTTP 响应中 HTTP 函数发送的最大数据量是 10 MB.我在 Google Storage Bucket 中有 100 MB 的文件，我想将这些文件从 Cloud Function 流式传输到我的应用程序.由于我们架构的安全方面，我的应用程序无法直接从存储桶中读取数据.

The Google Cloud Function's quota page: https://cloud.google.com/functions/quotas says that the Maximum amount of Data sent by HTTP Function in an HTTP Response is 10 MB. I have files of 100 MB in Google Storage Bucket which I want to stream from the Cloud Function to my application. Due to security aspects of our architecture, my application cannot read directly from the Storage Bucket.

云函数中的示例 Go 代码是:

A Sample Go Code in the Cloud Function is:

func MyFn(w http.ResponseWriter, r *http.Request) {

    var appendedBytes []byte
    for {
        decryptedFileBuffer, err := fh.GetNextDecryptedPage()
        if err != nil {
            break
        }
        appendedBytes = append(appendedBytes, decryptedFileBuffer...)
    }
    w.Write(appendedBytes)
}

如果我尝试使用 Cloud Function 获取超过 10 MB 的文件，则会出现以下错误:

If I try to get files more than 10 MB using Cloud Function, I get following errors:

错误:不正确的函数响应.函数调用是打断了.

Error: incorrect function response. Function invocation was interrupted.

错误:无法处理请求

如何从 Cloud Function 获取超过 10 MB 的文件?

How can I get files from than 10 MB from the Cloud Function?

推荐答案

当我们查看配额页面时，我们看到一个否反对是否可以增加配额.镇上的一个新玩家是 Google Cloud Run……请参阅此处.这提供了与 Cloud Functions 非常相似的功能，但使用 Kubernetes 作为函数托管环境.quota 限额为 32MB ...但这也低于您的需要，而且也是标记为no以增加.

When we look at the quota page, we see a no against whether or not the quota can be increased. A new player in town is Google Cloud Run ... see here. This provides very similar capabilities to Cloud Functions but uses Kubernetes as the function hosting environment. It's quota allowance is 32MB ... but this too is less than what you need and is also flagged as no for increase.

我的一个直接想法是用便利(一个简单的云功能)换取灵活性，并在 GCP 上运行您自己的计算资源.例如，运行 Compute Engines、App Engines 或 Kubernetes 集群，其中包含的应用程序通过 Web 服务器提供响应是一个完美的提议，并且不受吞吐量限制.您必须对计费进行数学计算以确定成本差异，因为指标会有所不同，但从技术角度来看，游戏会发生巨大变化.云功能/云运行功能主要用于支持非常短的微服务，我猜测直接传输 100MB 不被视为微服务.它也可能不像人们想象的那样具有成本效益.据我了解，Cloud Functions 按使用时间单位计费.传输 100MB 可能是缓慢的延迟提议，与我们想象的始终运行计算相比，可能并不具有成本效益.

An immediate thought that occurs to me is to trade convenience (a simple cloud function) for flexibility and run your own compute resources on GCP. For example, running up Compute Engines, App Engines or a Kubernetes cluster where the contained application serves responses through a Web Server is a perfectly fine proposition and not constrained in throughtput. You will have to do the math on billing to determine the cost differentials as the metrics will be different but from a technical perspective, the game changes dramatically. The cloud functions/cloud run capabilities are there primarily to support very short natured microservices and I am guessing that the transmission of 100MB directly is not considered a microservice. It also may not be as cost effective as one may think. It is my understanding that Cloud Functions bills per unit of time utilized. Transmitting 100MBs is likely to be slow latency proposition and may not be as cost effective compared to always running compute as we may think.

这篇关于Cloud Function - 获取超过 10 MB 的文件内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！