您如何并行运行功能? [英] How do you run functions in parallel?
问题描述
我的愿望是基于一些自定义的 select 语句从数据库中检索x条记录，输出将是json数据数组.然后，我想将数组中的每个元素并行传递给另一个lambda函数.
My desire is to retrieve x number of records from a database based on some custom select statement, the output will be an array of json data. I then want to pass each element in the array into another lambda function in parallel.
因此，如果返回1000条记录，则需要并行执行1000个lambda函数(我将帐户限制增加到所需的数量).如果每千个记录中有30个失败，则检索记录的主要任务需要了解它.
So if 1000 records are returned, 1000 lambda functions need to be executed in parallel (I increase my account limit to what I need). If 30 out of 1000 fail, the main task that was retrieving the records needs to know about it.
我正在努力把这个简单的流程放在一起.
I'm struggling to put together this simple flow.
我目前使用javascript和AWS Aurora.我不是在寻找可以检索数据的node.js/javascript代码，只是在寻找AWS Step Functions配置以及如何在每个函数中构建数组.
I currently use javascript and AWS Aurora. I'm not looking for node.js/javascript code that retrieves the data, just the AWS Step Functions configuration and how to build an array within each function.
谢谢.
推荐答案
如果返回1000条记录，则需要1000个lambda函数并行执行
if 1000 records are returned, 1000 lambda functions need to be executed in parallel
Step Functions不支持您要实现的目标.无法根据收到的输入来修改状态机任务.因此，例如，不能将并行任务配置为基于在数组输入中接收到的项目数来添加/删除函数.
What you are trying to achieve is not supported by Step Functions. A State Machine task cannot be modified based on the input it received. So for instance, a Parallel task cannot be configured to add/remove functions based on the number of items it received in an array input.
您可能应该考虑使用 SQS Lambda触发器.从数据库检索到的记录数可以添加到SQS队列中，这将为接收到的每个项目触发Lambda函数.
You should probably consider using SQS Lambda trigger. Number of records retrieved from DB can be added to SQS queue which will then trigger a Lambda function for each item received.
如果每千个记录中有30个失败，则检索记录的主要任务需要了解它.
If 30 out of 1000 fail, the main task that was retrieving the records needs to know about it.
有多种方法可以实现此目的.如果Lambda返回错误，则SQS不会从队列中删除项目.您可以配置 DLQ 和 RedrivePolicy .或者，您可能想提出一个自定义解决方案，以确保失败的Lambda能够调用从数据库获取记录的服务.
There are various ways to achieve this. SQS won't delete an item from the queue if Lambda returns an error. You can configure DLQ and RedrivePolicy based on your requirements. Or you may want to come up with a custom solution to keep the count on failing Lambdas to invoke the service that fetch records from the DB.
这篇关于您如何并行运行功能?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
