Azure Functions的吞吐量低下 [英] Poor throughput on Azure Functions

问题描述

我正在评估Azure Functions以创建缩略图.较大图像的源URL放置在存储队列中，带有队列触发器的C#函数用于处理URL(从源下载，调整大小并上传到另一个位置).

I'm evaluating Azure Functions to create thumbnail images. The source URLs of larger images are placed in a storage queue, and a C# function with a queue trigger is used to process the URLs (download from source, resize, and upload to another location).

每个函数调用都需要500毫秒以内的时间进行处理，这很好.但是，在运行一堆测试之后，我发现整体并行处理吞吐量并不那么好.队列中的工作负载为1500-2000个项目，该平台每秒仅执行约10个功能实例.

Each function call takes under 500ms for processing, which is fine. However, after running a bunch of tests, I've found that the overall parallel processing throughput is not that great. With workloads of 1500-2000 items in the queue, the platform only executes around 10 function instances per second.

有什么方法可以扩展并让平台同时执行更多的功能实例?

Is there any way to scale out and make the platform execute more function instances concurrently?

推荐答案

在消耗(动态)计划下运行时，当我们看到您的功能是跟不上.这种横向扩展不是瞬时的，因此可能是您的测试在添加更多实例之前或之后不久结束，才可以看到这些新实例的效果.在应用服务(经典)计划中运行时，您可以预先控制实例数量，并且可以扩展到所需数量.

When running under the Consumption (Dynamic) plan, the system will scale out to more instances automatically when we see that your function is not keeping up. This scale out is not instantaneous, so it might be that your test concluded before or shortly after more instances were added before the effects of those new instances could be seen. When running in an App Service (Classic) plan, you control the number of instances up front and can scale out to the number you require.

您可以在host.json文件中为队列设置一些配置旋钮，这些旋钮会影响每个Function App实例的并行度.在queues配置部分下，您可以设置batchSize和newBatchThreshold，例如:

There are a few configuration knobs for queues that you can set in your host.json file that affect the amount of parallelism per Function App instance. Under the queues configuration section, you can set batchSize and newBatchThreshold, e.g.:

{
   "queues": {
      "batchSize": 32,
      "newBatchThreshold": 50
    }
}

batchSize是每次提取时从队列中拉出的消息数.然后，批量处理所有消息. newBatchThreshold控制何时提取下一批消息.仅当当前正在处理的消息数降至此阈值以下时，才从队列中提取新的消息批次.因此，增加newBatchThreshold将允许并行处理更多消息.有关这些设置的更多详细信息，请参见此处.

batchSize is the number of messages that are pulled from the queue on each fetch. All the messages in a batch are then processed in parallel. newBatchThreshold governs when the next batch of messages will be fetched. A new batch of messages is only fetched from the queue when the number of messages currently being processed drops below this threshold. So increasing newBatchThreshold will allow more messages to be processed in parallel. See the wiki here for more details on these settings.

请注意，调整这些设置时必须考虑工作量.例如，如果您的函数占用大量内存/CPU，则不能在单个实例上并行运行太多函数.因此，您可能需要尝试一下.

Note that you have to take your workload into account when adjusting these settings. For example, if your function is very memory/CPU intensive, you can't run too many of them in parallel on a single instance. So you may have to experiment a bit.

除上述所有功能外，还应确保您的函数是正确的async函数，以确保不会在IO上不必要地阻塞线程，并且函数中不存在其他潜在的瓶颈代码本身.

In addition to all of the above, you should also make sure your function is a proper async function, to ensure threads aren't being blocked unnecessarily on IO, and that no other potential bottle-necks exist in your function code itself.

这篇关于Azure Functions的吞吐量低下的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！