来自 API 的 node.js pg-promise 和分页 [英] node.js pg-promise and pagination from API

问题描述

看https://github.com/vitaly-t/pg-promise/wiki/Data-Imports 有关于如何使用它进行导入的非常详细的文档.

Looking at https://github.com/vitaly-t/pg-promise/wiki/Data-Imports there's a very detailed doc on how to use it for importing.

然而，虽然这适用于演示场景，但我不知道如何将其应用于我的案例.

However while that works for the demoed scenario I don't know how to apply it on my case.

当我进行网络调用时，我获得了实际的 JSON 数据和标题中的参数，它为我提供了下一页的值(可以是日期、字符串或数字值).

When I do my web call, I get the actual JSON data and a paramter in the header which gives me a value for the next page (could be a date or String or a number value).

在示例中，它说:

db.tx('massive-insert', t => {
    return t.sequence(index => {
        return getNextData(index)
            .then(data => {
                if (data) {
                    const insert = pgp.helpers.insert(data, cs);
                    return t.none(insert);
                }
            });
    });
})
    .then(data => {
        console.log('Total batches:', data.total, ', Duration:', data.duration);
    })
    .catch(error => {
        console.log(error);
    });

在这种情况下，sequence(index 将使用似乎递增 +1 的索引.但就我而言，

In this case, sequence(index will use index which seems to increment +1. But in my case,

function getNextData(nextPage) {
    //get the data for nextPage
    .....
   //get the nextPage if exists for future use
   nextPage = response.next;

   resolve(data);
}

我的问题是，在这个例子中，如何用 nextPage 替换 index，因为每个新的 Promise 都需要使用上一个的 nextPage一个.

My question is, how can I replace index with nextPage in this example, as each new Promise needs to use the nextPage from previous one.

后期如果我想从 nextPageInfo 的某个值中获取信息?

LATER And if I want to fetch info from a certain value of nextPageInfo?

例如:

db.any('Select value from table')
      .then(function(value) {

var data = value; //not working

db.tx('massive-insert', t => {
    return t.sequence((index, data) => {
        return getNextData(index, data)
            .then(a => {
                if (a) {
                    const insert = pgp.helpers.insert(a.data, cs);
                    return t.none(insert).then(() => a.nextPageInfo);
                }
            })
    });
})
    .then(data => {
        // COMMIT has been executed
        console.log('Total batches:', data.total, ', Duration:', data.duration);
    })
    .catch(error => {
        // ROLLBACK has been executed
        console.log(error);
    })

}

推荐答案

针对这个问题，我扩展了文章 数据导入nofollow noreferrer">extras 部分，它为您提供了所需的示例.从文章复制的例子:

Following this question, I have extended article Data Imports with the new extras section, which gives you exactly the example that you need. The example copied from the article:

function getNextData(t, index, nextPageInfo) {
    // t = database transaction protocol

    // NOTE: nextPageInfo = undefined when index = 0

    return new Promise((resolve, reject) {

        /* pull the next data, according to nextPageInfo */            

        /* do reject(error) on an error, to ROLLBACK changes */
    
        if(/* there is still data left*/) {
            // if whateverNextDetails = undefined, the data will be inserted,
            // but the sequence will end there (as success).
            resolve({data, nextPageInfo: whateverNextDetails});
        } else {
            resolve(null);
        }   
    });
}

db.tx('massive-insert', t => {
    return t.sequence((index, data) => {
        return getNextData(t, index, data)
            .then(a => {
                if (a) {
                    const insert = pgp.helpers.insert(a.data, cs);
                    return t.none(insert).then(() => a.nextPageInfo);
                }
            })
    });
})
    .then(data => {
        // COMMIT has been executed
        console.log('Total batches:', data.total, ', Duration:', data.duration);
    })
    .catch(error => {
        // ROLLBACK has been executed
        console.log(error);
    });

请注意，由于我们将 getNextData 的结果链接到 nextPageInfo 的值，那么如果它的值为 undefined，它将执行下一个插入，但随后将结束序列(成功).

Please note that since we are chaining the result from getNextData to the value of nextPageInfo, then if its value is undefined, it will do the next insert, but then will end the sequence (as success).

这篇关于来自 API 的 node.js pg-promise 和分页的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！