Lambda:是否有任何批处理调度程序? [英] Lambda : Is any Batch processing scheduler available?
问题描述
问题:从Dynamo DB中获取2000个项目，并逐批处理(从100个项目创建POST请求)(批量大小= 100).
Problem : Fetch 2000 items from Dynamo DB and process(Create a POST req from 100 items) it batch by batch (Batch size = 100).
问题:无论如何，我可以通过AWS中的任何配置来实现它.
Question : Is there anyway that I can achieve it from any configuration in AWS.
PS:我已经配置了cron计划来运行我的Lambda函数.我正在使用Java.我已经制作了同步执行的多线程应用程序，但这最终极大地增加了我的计算时间.
PS : I've configured a cron schedule to run my Lambda function. I'm using Java. I've made multi-threaded application which synchronously does so, but this eventually increases my computation time drastically.
推荐答案
我有相同的问题，并考虑通过以下方式解决.如果您尝试的话，请告诉我.
I have the same problem and thinking of solving it in following way. Please let me know if you try it.
-
计划作业，使用Lambda函数从DynamoDB中获取N个项目
Schedule Job to fetch N items from DynamoDB using Lambda function
Lambda函数将向SQS提交M条消息以处理每条消息 item和触发器lambda函数，在这种情况下，应调用 Lambda函数M次每个Lambda函数将处理请求 消息中给出
Lambda function in #1 will submit M messages to SQS to process each item and trigger lambda functions, in this case it should call lambda functions M times Each lambda function will process request given in the message
为了实现此目的，您需要通过CloudWatch安排事件，设置SQS并创建由SQS事件触发的lambda函数.
In order to achieve this you need to schedule an event via CloudWatch, setup SQS and create lambda function triggered by SQS events.
老实说，我不确定这是否具有价格效益，但应该可以使用.假设您的提取大小太小，这应该是合理的.
Honestly, I am not sure if this is price effective but it should be working. Assuming your fetch size is so low, this should be reasonable.
在这种情况下，您也可以尝试使用SNS，而无需担心SQS消息轮询.
Also you can try using SNS in this case you don't need to worry about SQS message polling.
这篇关于Lambda:是否有任何批处理调度程序?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
