如何使用AWS JavaScript SDK（dynamoDB）处理UnprocessedItems？ [英] How to handle UnprocessedItems using AWS JavaScript SDK (dynamoDB)?

问题描述

我正在尝试使用AWS Lambda函数来处理来自SendGrid的事件。据我了解，该事件将是一个包含可变数量的JSON对象的数组，每个对象代表一个给定的事件。我想使用batchWriteItem将这些事件写入DynamoDB并循环该过程，直到我没有返回任何UnprocessedItems。但是，我陷入了无限循环。这是我现在的代码：

I am trying to use an AWS Lambda function to handle events from SendGrid. As I understand it, the event will be an array with a variable number of JSON objects, each representing a given event. I want to write these events to DynamoDB using batchWriteItem and loop the process until I'm not returned any UnprocessedItems. However, I'm getting stuck in an infinite loop. Here is my code now:

console.log('Loading function');

var aws = require('aws-sdk');
var dynamo = new aws.DynamoDB();
params = {};

exports.handler = function(sg_event, context) {

    var items = [];
    for(var i = 0; i < sg_event.length; i++) {
        var obj = sg_event[i];
        var request = {
            PutRequest: {
                Item: {
                    email: { S: obj.email },
                    timestamp: { S: obj.timestamp.toString() },
                    sg_message_id: { S: obj.sg_message_id },
                    event: { S: obj.event }
                }
            }
        };
        items.push(request);
    }

    params = {
        RequestItems: {
            sendgrid_response: items
        }
    }

    do {
        dynamo.batchWriteItem( params, function(err, data) {
            if(err)
                context.fail(err);
            else
                params.RequestItems = data.UnprocessedItems;
        });
    } while(!isEmpty(params.RequestItems));
};

function isEmpty(obj) {
    return (Object.keys(obj).length === 0);
}

我认为问题是试图在回调函数中设置params，但是我不知道我应该怎么做...我知道我可以在原始回调中使用UnprocessedItems调用另一个batchWriteItem，但我仍然需要能够多次运行该函数需要确保编写所有UnprocessedItems。如何正确循环batchWriteItem？

I think the problem is trying to set the params in the callback function, but I don't know how else I'm supposed to do it...I know I could just call another batchWriteItem using UnprocessedItems within the callback of the original one, but I still need to be able to run the function as many times as needed to ensure all UnprocessedItems are written. How can I loop the batchWriteItem correctly?

推荐答案

@Daniela Miao，感谢您分享解决方案。

@Daniela Miao, Thanks for sharing the solution.

我们可以在您发布的代码中添加一个代码块，以避免DynamoDB发生异常。这将检查 params.RequestItems 是否在请求DynamoDB再次批量写入之前有未处理的数据。

We can add one code block in your posted code which will avoid exception from DynamoDB. This will check if params.RequestItems has Unprocessed data before requesting the DynamoDB for batch write again.

//db is AWS.DynamoDB Client
var processItemsCallback = function(err, data) {
  if (err) { 
     //fail
  } else {
    var params = {};
    params.RequestItems = data.UnprocessedItems;
    /*
    * Added Code block 
    */
    if(Object.keys(params.RequestItems).length != 0) {
      db.batchWriteItem(params, processItemsCallback);
    }
  }
};

db.batchWriteItem(/*initial params*/, processItemsCallback);

这篇关于如何使用AWS JavaScript SDK（dynamoDB）处理UnprocessedItems？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！