使用外部 API 调用和 findOneAndUpdate 循环结果 [英] Looping Results with an External API Call and findOneAndUpdate

问题描述

我正在尝试编写一个程序，该程序使用 mongoose 从 mongo 数据库中获取文档并使用 API 处理它们，然后使用处理结果编辑数据库中的每个文档.我的问题是我有问题，因为我不完全理解 nodejs 和异步.这是我的代码:

I am trying to write a program that gets the documents from a mongo database with mongoose and process them using an API and then edits each document in the database with the results of the processing. My problem is that I have problems because I don't understand completely nodejs and the asynchronous. This is my code:

Model.find(function (err, tweets) {
    if (err) return err;
    for (var i = 0; i < tweets.length; i++) {
        console.log(tweets[i].tweet);
        api.petition(tweets[i].tweet)
            .then(function(res) {
                TweetModel.findOneAndUpdate({_id: tweets[i]._id}, {result: res}, function (err, tweetFound) {
                    if (err) throw err;
                    console.log(tweetFound);
                });
            })
            .catch(function(err) {
                console.log(err);
            })
    }
})

问题是在 findOneAndUpdate 中，tweets 是未定义的，所以它找不到那个 id.有什么解决办法吗?谢谢

The problem is that in the findOneAndUpdate, tweets is undefined so it can't find that id. Any solution? Thanks

推荐答案

你真正缺少的核心是 Mongoose API 方法也使用 "Promises"，但您似乎只是从文档或使用回调的旧示例中复制.解决方案是转换为仅使用 Promises.

The core thing you are really missing is that the Mongoose API methods also use "Promises", but you seem to just be copying from documentation or old examples using callbacks. The solution to this is to convert to using Promises only.

Model.find({},{ _id: 1, tweet: 1}).then(tweets => 
  Promise.all(
    tweets.map(({ _id, tweet }) => 
      api.petition(tweet).then(result =>   
       TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
         .then( updated => { console.log(updated); return updated })
      )
    )
  )
)
.then( updatedDocs => {
  // do something with array of updated documents
})
.catch(e => console.error(e))

除了回调的一般转换之外，主要的变化是使用 Promise.all() 解析来自 Array.map() 正在处理 .find() 而不是 for 循环.这实际上是您尝试中最大的问题之一，因为 for 实际上无法控制异步函数何时解析.另一个问题是混合回调"，但这就是我们在这里通常只使用 Promise 来解决的问题.

Aside from the general conversion from callbacks, the main change is using Promise.all() to resolve the ouput from the Array.map() being processed on the results from .find() instead of the for loop. That is actually one of the biggest problems in your attempt, since the for cannot actually control when the async functions resolve. The other issue is "mixing callbacks", but that is what we are generally addressing here by only using Promises.

在 Array 中.map() 我们从 API 调用返回 Promise，链接到 findOneAndUpdate() 这实际上是在更新文档.我们还使用 new: true 来实际返回修改后的文档.

Within the Array.map() we return the Promise from the API call, chained to the findOneAndUpdate() which is actually updating the document. We also use new: true to actually return the modified document.

Promise.all() 允许一个Promise 数组"；解析并返回结果数组.您看到的这些是 updatedDocs.这里的另一个优点是内部方法将以并行"方式触发.而不是系列.这通常意味着更快的解决方案，但需要更多的资源.

Promise.all() allows an "array of Promise" to resolve and return an array of results. These you see as updatedDocs. Another advantage here is that the inner methods will fire in "parallel" and not in series. This usually means a faster resolution, though it takes a few more resources.

还要注意，我们使用了投影"{ _id: 1, tweet: 1 } 只从 Model.find() 结果，因为这些是其余调用中唯一使用的.当您不使用其他值时，这可以节省为每个结果返回整个文档.

Note also that we use the "projection" of { _id: 1, tweet: 1 } to only return those two fields from the Model.find() result because those are the only ones used in the remaining calls. This saves on returning the whole document for each result there when you don't use the other values.

您可以简单地返回Promise 来自 findOneAndUpdate()，但我只是在 console.log() 中添加，因此您可以看到此时输出正在触发.

You could simply just return the Promise from the findOneAndUpdate(), but I'm just adding in the console.log() so you can see the output is firing at that point.

正常的生产使用应该没有它:

Normal production use should do without it:

Model.find({},{ _id: 1, tweet: 1}).then(tweets => 
  Promise.all(
    tweets.map(({ _id, tweet }) => 
      api.petition(tweet).then(result =>   
       TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
      )
    )
  )
)
.then( updatedDocs => {
  // do something with array of updated documents
})
.catch(e => console.error(e))

另一个调整"可能是使用蓝鸟"Promise.map() 的实现，其中两者结合了常见的 Array.map() 到 Promise(s) 实现具有控制并发"的能力；运行并行调用:

Another "tweak" could be to use the "bluebird" implementation of Promise.map(), which both combines the common Array.map() to Promise(s) implementation with the ability to control "concurrency" of running parallel calls:

const Promise = require("bluebird");

Model.find({},{ _id: 1, tweet: 1}).then(tweets => 
  Promise.map(tweets, ({ _id, tweet }) => 
    api.petition(tweet).then(result =>   
      TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
    ),
    { concurrency: 5 }
  )
)
.then( updatedDocs => {
  // do something with array of updated documents
})
.catch(e => console.error(e))

并行"的替代方法将按顺序执行.如果太多的结果导致太多的 API 调用和写回数据库的调用，可能会考虑这样做:

An alternate to "parallel" would be executing in sequence. This might be considered if too many results causes too many API calls and calls to write back to the database:

Model.find({},{ _id: 1, tweet: 1}).then(tweets => {
  let updatedDocs = [];
  return tweets.reduce((o,{ _id, tweet }) => 
    o.then(() => api.petition(tweet))
      .then(result => TweetModel.findByIdAndUpdate(_id, { result }, { new: true })
      .then(updated => updatedDocs.push(updated))
    ,Promise.resolve()
  ).then(() => updatedDocs);
})
.then( updatedDocs => {
  // do something with array of updated documents
})
.catch(e => console.error(e))

在那里我们可以使用 Array.reduce() 到链"；承诺一起允许他们按顺序解决.请注意，结果数组保留在范围内，并使用附加到连接链末尾的最终 .then() 换出，因为您需要这种技术来收集"结果.Promise 在该链"中的不同点解析的结果.

There we can use Array.reduce() to "chain" the promises together allowing them to resolve sequentially. Note the array of results is kept in scope and swapped out with the final .then() appended to the end of the joined chain since you need such a technique to "collect" results from Promises resolving at different points in that "chain".

在现代环境中，从 NodeJS V8.x 开始，它实际上是当前的 LTS 版本并且已经有一段时间了，您实际上已经支持 async/await.这使您可以更自然地编写流程

In modern environments as from NodeJS V8.x which is actually the current LTS release and has been for a while now, you actually have support for async/await. This allows you to more naturally write your flow

try {
  let tweets = await Model.find({},{ _id: 1, tweet: 1});

  let updatedDocs = await Promise.all(
    tweets.map(({ _id, tweet }) => 
      api.petition(tweet).then(result =>   
        TweetModel.findByIdAndUpdate(_id, { result }, { new: true })
      )
    )
  );

  // Do something with results
} catch(e) {
  console.error(e);
}

如果资源有问题，甚至可能按顺序处理:

Or even possibly process sequentially, if resources are an issue:

try {
  let cursor = Model.collection.find().project({ _id: 1, tweet: 1 });

  while ( await cursor.hasNext() ) {
    let { _id, tweet } = await cursor.next();
    let result = await api.petition(tweet);
    let updated = await TweetModel.findByIdAndUpdate(_id, { result },{ new: true });
    // do something with updated document
  }

} catch(e) {
  console.error(e)
}

还要注意 findByIdAndUpdate() 也可以被用作匹配 _id 已经是隐含的，所以你不需要整个查询文档作为第一个参数.

Noting also that findByIdAndUpdate() can also be used as matching the _id is already implied so you don't need a whole query document as a first argument.

最后一点，如果您实际上根本不需要更新的文档作为响应，那么 bulkWrite() 是更好的选择，它允许写入通常在单个请求中在服务器上处理:

As a final note if you don't actually need the updated documents in response at all, then bulkWrite() is the better option and allows the writes to generally process on the server in a single request:

Model.find({},{ _id: 1, tweet: 1}).then(tweets => 
  Promise.all(
    tweets.map(({ _id, tweet }) => api.petition(tweet).then(result => ({ _id, result }))
  )
).then( results =>
  Tweetmodel.bulkWrite(
    results.map(({ _id, result }) => 
      ({ updateOne: { filter: { _id }, update: { $set: { result } } } })
    )
  )
)
.catch(e => console.error(e))

或者通过 async/await 语法:

try {
  let tweets = await Model.find({},{ _id: 1, tweet: 1});

  let writeResult = await Tweetmodel.bulkWrite(
    (await Promise.all(
      tweets.map(({ _id, tweet }) => api.petition(tweet).then(result => ({ _id, result }))
    )).map(({ _id, result }) =>
      ({ updateOne: { filter: { _id }, update: { $set: { result } } } })
    )
  );
} catch(e) {
  console.error(e);
}

上面显示的几乎所有组合都可以作为 bulkWrite() 方法接受一个数组"；指令，因此您可以从上述每个方法的已处理 API 调用构建该数组.

Pretty much all of the combinations shown above can be varied into this as the bulkWrite() method takes an "array" of instructions, so you can construct that array from the processed API calls out of every method above.

这篇关于使用外部 API 调用和 findOneAndUpdate 循环结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！