循环中的节点JS Api请求 [英] Node JS Api request in loop

问题描述

我正在尽我最大的努力来避免使用Node JS造成回调地狱.但是我正在尝试进行大量的api请求，并将其插入到我的数据库中.

I'm trying my damndest to avoid callback hell with my Node JS. But I'm trying to make a large number of api-requests and insert these into my database.

我的问题(当然)是我的for循环在完成请求和数据库插入之前运行并递增i.

My issue here (of course) is that my for-loop runs and increments i before I finish my request and database insertion.

for(var i = 0; i <= 1 ; i++){
    apiRequest = data[i]; 
    apicall(apiRequest);
}


function apicall(urlApi){
    request((urlApi), function(error, response, body){
        if(error){
            console.log("error");
        } else if(!error && response.statusCode == 200){
            var myobj = JSON.parse(body);
            dbInsert(myobj);
        }
    });
}

function dbInsert(obj) {
    //insert into database
}

如果其他人还会遇到这个问题，我可以真正推荐这篇博文，该博文是在阅读 joshvermaire 的回复后找到的:

If someone else would come by this question I can truly recommend this blogpost which I found after reading the response by joshvermaire:

http://www.sebastianseilund.com/nodejs-async-in-practice

推荐答案

有很多方法可以解决此类问题.首先，如果您可以并行运行所有API调用(同时在运行中)，并且将它们插入数据库的顺序无关紧要，那么这样做可以更快地得到结果(vs .按顺序对其进行序列化.)

There are a number of ways to approach this type of problem. Firstly, if you can run all the API calls in parallel (all in flight at the same time) and it doesn't matter what order they are inserted in your database, then you can get a result a lot faster by doing that (vs. serializing them in order).

在下面的所有选项中，您将使用以下代码:

In all the options below, you would use this code:

const rp = require('request-promise');

function apicall(urlApi){
    return rp({url: urlApi, json: true}).then(function(obj){
        return dbInsert(obj);
    });
}

function dbInsert(obj) {
    //insert into database
    // return a promise that resolves when the database insertion is done
}

并行使用ES6标准承诺

let promises = [];
for (let i = 0; i <= data.length; i++) {
    promises.push(apicall(data[i]));
}

Promise.all(promises).then(() => {
    // all done here
}).catch(err => {
    // error here
});

使用Bluebird Promise库进行并行

使用Bluebird Promise库，您可以使用Promise.map()来迭代您的数组，并且可以将其传递给concurrency选项以控制同时进行的异步调用数量，这可以避免使数据库不堪重负或目标API主机，并可能有助于控制最大内存使用情况.

With the Bluebird Promise library, you can use Promise.map() to iterate your array and you can pass it the concurrency option to control how many async calls are in flight at the same time which might keep from overwhelming either the database or the target API host and might help control max memory usage.

Promise.map(data, apiCall, {concurrency: 10}).then(() => {
    // all done here
}).catch(err => {
    // error here
});

使用标准ES6承诺进行批量生产

如果由于某种原因(例如按顺序插入数据库)必须对它们进行序列化，则可以这样做.下面显示的.reduce()模式是使用标准ES6序列化数组上的promise操作的经典方法:

If you have to serialize them for some reason such as inserting into the database in order, then you can do that like this. The .reduce() pattern shown below is a classic way to serialize promise operations on an array using standard ES6:

data.reduce(data, (p, item) => {
    return p.then(() => {
        return apicall(item);
    });
}, Promise.resolve()).then(() => {
    // all done here
}).catch(err => {
    // error here
});

使用Bluebird承诺的系列作品

Bluebird有一个Promise.mapSeries()，它可以依次迭代一个数组，并调用一个函数，该函数对数组中的每个项目都返回一个promise，这比手动完成要简单一些.

Bluebird has a Promise.mapSeries() that iterates an array in series, calling a function that returns a promise on each item in the array which is a little simpler than doing it manually.

Promise.mapSeries(data, apiCall).then(() => {
    // all done here
}).catch(err => {
    // error here
});

这篇关于循环中的节点JS Api请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！