来自分页 API 的过滤 axios 结果数组为空 [英] Array of filtered axios results from paginated API is empty

问题描述

在我下面的代码中，我的 console.log(response) 上有一个空数组，但 console.log(filterdIds) 在 getIds 功能正在显示我想要的数据.我觉得我的决心不对.

请注意，我运行了一次 do..while 进行测试.API 是分页的.如果记录来自昨天，它将继续进行，如果不是，则 do..while 停止.

有人能指出我正确的方向吗?

const axios = require(axios");函数 getToken() {//获取令牌}函数 getIds(jwt) {返回新的承诺((解决)=> {让 pageNumber = 1;const filterdIds = [];常量配置 = {//配置东西};做 {轴(配置).then((响应) => {response.forEach(元素 => {//一些逻辑，如果为真则:filterdIds.push(element.id);控制台日志(filterdIds)；});}).catch(错误=> {控制台日志(错误)；});} while (pageNumber != 1)解决(filterdIds)；});}获取令牌().then(token => {返回令牌；}).then(jwt => {返回 getIds(jwt);}).then(响应 => {控制台日志(响应)；}).catch(错误=> {控制台日志(错误)；});

由于 do..while，我也不知道将拒绝放在 getIds 函数中的什么位置.

解决方案

根本问题是 resolve(filterdIds); 在请求触发之前同步运行，所以它保证为空.>

Promise.all 或 Promise.allSettled 如果您知道需要多少页面(或者如果您使用块大小来发出多个请求)可以提供帮助——稍后会详细介绍).这些方法并行运行.这是一个可运行的概念验证示例:

const pages = 10;//用于运行循环的某些页面值公理.get("https://httpbin.org")//一些初始请求，如 getToken.then(response =>//响应有标记，为简单起见忽略承诺.all(Array(pages).fill().map((_, i) =>//制作一个请求承诺数组axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${i + 1}`)))).then(响应 => {//对响应数据执行过滤/减少const 结果 = response.flatMap(response =>响应数据.filter(e => e.id % 2 === 0)//一些愚蠢的过滤器.map(({id, name}) => ({id, name})));//使用结果控制台日志(结果)；}).catch(err => console.error(err));

<script src="https://unpkg.com/axios/dist/axios.min.js"></脚本>

网络选项卡显示并行发生的请求:

如果页面数量未知，并且您打算一次触发一个请求，直到您的 API 通知您页面结束，顺序循环很慢​​但可以使用.这种策略的异步/等待更干净:

(async() => {//像 getToken;应该处理错误const tokenStub = await axios.get("https://httpbin.org");常量结果 = [];//page += 10 使代码片段运行得更快；你可能会使用 page++for (让页面 = 1;; 页面 += 10) {尝试 {const url = `https://jsonplaceholder.typicode.com/comments?postId=${page}`;const response = await axios.get(url);//检查您的 API 发送的任何条件，告诉您没有更多页面if (response.data.length === 0) {休息;}for(response.data 的常量注释){if (comment.id % 2 === 0) {//一些愚蠢的过滤器const {name, id} = 评论；结果.push({name, id});}}}catch (err) {//到达页面末尾或其他一些错误休息;}}//使用结果控制台日志(结果)；})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></脚本>

这是顺序请求瀑布:

如果您想增加并行度，可以使用任务队列或分块循环.分块循环将结合这两种技术一次请求 n 条记录，并检查块中的每个结果是否存在终止条件.这是一个去掉过滤操作的简单示例，它是异步请求问题的附带条件，可以在响应到达后同步完成:

(async() => {常量结果 = [];常量块 = 5;for (let page = 1;; page += chunk) {尝试 {const 响应 = 等待 Promise.all(数组(chunk).fill().map((_, i) =>axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${page + i}`)));for(响应的常量响应){for(response.data 的常量注释){const {name, id} = 评论；结果.push({name, id});}}//检查结束条件if (responses.some(e => e.data.length === 0)) {休息;}}抓住(错误){休息;}}//使用结果控制台日志(结果)；})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></脚本>

(上图是除了100个请求，但是可以看到一次5的块大小)

请注意，这些代码片段是概念验证，可以减少不加选择地捕获错误，确保捕获所有抛出等.将其分解为子函数时，请确保 .then 和 await 调用者中的所有承诺——不要试图把它变成同步代码.

另见

• 如何从异步返回响应调用? 和 为什么我的变量在函数内部修改后没有改变?- 异步代码参考，解释了为什么数组为空.
• 什么是显式承诺构造反模式以及如何避免它?，它警告不要添加 new Promise 来帮助解决已经返回承诺的代码.

In my code below I get an empty array on my console.log(response) but the console.log(filterdIds) inside the getIds function is showing my desired data. I think my resolve is not right.

Note that I run do..while once for testing. The API is paged. If the records are from yesterday it will keep going, if not then the do..while is stopped.

Can somebody point me to the right direction?

const axios = require("axios");

function getToken() {
    // Get the token
}

function getIds(jwt) {
    return new Promise((resolve) => {
        let pageNumber = 1;
        const filterdIds = [];

        const config = {
            //Config stuff
        };

        do {
            axios(config)
                .then((response) => {
                    response.forEach(element => {
                        //Some logic, if true then:
                        filterdIds.push(element.id);
                        console.log(filterdIds);
                    });
                })
                .catch(error => {
                    console.log(error);
                });
        } while (pageNumber != 1)
        resolve(filterdIds);
    });
}


getToken()
    .then(token => {
        return token;
    })
    .then(jwt => {
        return getIds(jwt);
    })
    .then(response => {
        console.log(response);
    })
    .catch(error => {
        console.log(error);
    });

I'm also not sure where to put the reject inside the getIds function because of the do..while.

解决方案

The fundamental problem is that resolve(filterdIds); runs synchronously before the requests fire, so it's guaranteed to be empty.

Promise.all or Promise.allSettled can help if you know how many pages you want up front (or if you're using a chunk size to make multiple requests--more on that later). These methods run in parallel. Here's a runnable proof-of-concept example:

const pages = 10; // some page value you're using to run your loop

axios
  .get("https://httpbin.org") // some initial request like getToken
  .then(response => // response has the token, ignored for simplicity
    Promise.all(
      Array(pages).fill().map((_, i) => // make an array of request promisess
        axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${i + 1}`)
      )
    )
  )
  .then(responses => {
    // perform your filter/reduce on the response data
    const results = responses.flatMap(response =>
      response.data
        .filter(e => e.id % 2 === 0) // some silly filter
        .map(({id, name}) => ({id, name}))
    );
    
    // use the results
    console.log(results);
  })
  .catch(err => console.error(err))
;

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

The network tab shows the requests happening in parallel:

If the number of pages is unknown and you intend to fire requests one at a time until your API informs you of the end of the pages, a sequential loop is slow but can be used. Async/await is cleaner for this strategy:

(async () => {
  // like getToken; should handle err
  const tokenStub = await axios.get("https://httpbin.org");
  
  const results = [];
  
  // page += 10 to make the snippet run faster; you'd probably use page++
  for (let page = 1;; page += 10) {
    try {
      const url = `https://jsonplaceholder.typicode.com/comments?postId=${page}`;
      const response = await axios.get(url);
      
      // check whatever condition your API sends to tell you no more pages
      if (response.data.length === 0) { 
        break;
      }
      
      for (const comment of response.data) {
        if (comment.id % 2 === 0) { // some silly filter
          const {name, id} = comment;
          results.push({name, id});
        }
      }
    }
    catch (err) { // hit the end of the pages or some other error
      break;
    }
  }
  
  // use the results
  console.log(results);
})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

Here's the sequential request waterfall:

A task queue or chunked loop can be used if you want to increase parallelization. A chunked loop would combine the two techniques to request n records at a time and check each result in the chunk for the termination condition. Here's a simple example that strips out the filtering operation, which is sort of incidental to the asynchronous request issue and can be done synchronously after the responses arrive:

(async () => {  
  const results = [];
  const chunk = 5;
  
  for (let page = 1;; page += chunk) {
    try {
      const responses = await Promise.all(
        Array(chunk).fill().map((_, i) => 
          axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${page + i}`)
        )
      );
      
      for (const response of responses) {      
        for (const comment of response.data) {
          const {name, id} = comment;
          results.push({name, id});
        }
      }
      
      // check end condition
      if (responses.some(e => e.data.length === 0)) { 
        break;
      }
    }
    catch (err) {
      break;
    }
  }
  
  // use the results
  console.log(results);
})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

(above image is an except of the 100 requests, but the chunk size of 5 at once is visible)

Note that these snippets are proofs-of-concept and could stand to be less indiscriminate with catching errors, ensure all throws are caught, etc. When breaking it into sub-functions, make sure to .then and await all promises in the caller--don't try to turn it into synchronous code.

See also

• How do I return the response from an asynchronous call? and Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference which explain why the array is empty.
• What is the explicit promise construction antipattern and how do I avoid it?, which warns against adding a new Promise to help resolve code that already returns promises.

这篇关于来自分页 API 的过滤 axios 结果数组为空的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！