如何在Redis中将MongoDB文档存储为缓存并通过超时刷新 [英] How to store MongoDB documents in redis as cache and flush by timeout

问题描述

我有处理嵌套JSON文档的nodejs应用程序，如下所示:

I have nodejs application that handles nested JSON documents likes the following:

var post = {
  id: 123,
  title: 'Sterling Archer',    
  comments: [
    {text: 'Comment text', tags: ['tag1', 'tag2', 'tag3']},
    {text: 'Comment test', tags: ['tag2', 'tag5']}
  ]  
};

并将它们存储在MongoDB数据库中.我的文档需要经常更新，但是如您所知，由于其性质，MongoDB的写入速度非常慢.为了解决此问题，我决定将文档存储在Redis中，并在某个超时时间(例如1-2小时后)刷新到MongoDB.

And stores them in MongoDB database. My documents require frequent updates but as you know, MongoDB is very slow on writes due their nature. To resolve this problem I've decided to store documents in Redis and flush to MongoDB at some timeout (after 1-2 hours for example).

下面是我的更新方法的代码示例:

So here the code example of my updating approach:

var redis_cli = require("redis").createClient();

app.post('/update/:id', function (req, res, next) {

  var id = req.params.id
  redis_cli.get(id, function (err, document) {
      if (err) return next(err);

      // If there is no key, try to get from MongoDB
      if (!document) {
         DocumentModel.findOneByid(id, function (err, document) {
             if (err) return next(err);
             if (!document) return next(new Error("Document with id " + id + " does not exists"));

             // Document in MongoDB, so store in redis
             redis_cli.set(id, JSON.stringify(document), function (err) {
                  if (err) return next(err);

                  updateDocument(document);
             });                    
         });    
      } else {
          updateDocument(JSON.parse(document));
      }

      function updateDocument (document) {
          // Do some updates ...
          document.title = "Updated post title";

          // Store to redis
          redis_cli.set(id, JSON.strinfy(document), function (err) {
              if (err) return next(err);

              // Document updated successful
              return res.status(200).send('OK');
          });              
      }          
  });
});

我的第一个问题是我如何处理文档?我的方法有什么问题吗?

My first question is what do think about my approach to work with documents? Is there any problems with my approach?

第二个问题是如何将Redis中的文档刷新回mongodb并从redis中删除文档?我的目的如下:我只想在用户需要时才将文档存储在Redis中，因此，如果不使用他们的文档，则应将其存储在MongoDB中，而不是Redis中.

And the second question is how to flush documents in Redis back to mongodb and remove ones from redis? My purpose are the following: I want to store documents in redis only when necessary by my users so if the don't work with their document it should be stored in MongoDB instead Redis.

推荐答案

您的方法似乎很合理.

要刷新"文档，请在Redis中保留已排序文档ID的排序集，其得分设置为更新时间戳.定期每分钟，在该集合上执行一次ZRANGE处理，以获取旧"(例如，一个小时前的最新更新)文档ID，并为每个ID执行一次GET操作，将其写入Mongo，然后从DEL中删除该文档和ZREM有序集.

To "flush" the documents, keep a sorted set in Redis of the ids of documents that were updated, their scores set to the update timestamp. Periodically, e.g. every minute, do a ZRANGE on that set to get "old" (e.g. last update more than an hour ago) document ids, and for each id do a GET of the document, write it to Mongo, DEL the document and ZREM from the ordered set.

编辑未经测试，并且完全在伪Node.js中组成了代码示例:

EDIT untested and totally made up code samples in pseudo-Node.js:

 function updateDocument (document) {
      // Do some updates ...
      document.title = "Updated post title";

      // Store to redis
      multi = redis_cli.multi();
      multi.set(id, JSON.strinfy(document);
      multi.zadd('last_update', time(), id);
      multi.exec(), function (err, replies) {
          if (err) return next(err);

          // Document updated successful
          return res.status(200).send('OK');
      });              
  }

  // call this function periodically, e.g. every minute or so
  function flushOldDocuments () {
      fromTime = time()-3600;
      while (redis_cli.zcount('last_update', '-inf', fromTime) > 0) {
          id = redis_cli.zrangebyscore('last_update', '-inf', fromTime, false, 0, 1); // no scores, offset 0, limit 1 -> get the oldest document
          redis_cli.watch(id);
          Mongo.write(JSON.parse(redis_cli.get(id))); // or something like that
          multi = redis_cli.multi();
          multi.zrem('last_update', id);
          multi.del(id);
          multi.exec(), function(err, replies) {
              // if the watch fails the exec, no harm done and the document will be flushed in an hour
              ...
          };
      };
  }

这篇关于如何在Redis中将MongoDB文档存储为缓存并通过超时刷新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！