来自分页API的已过滤axios结果数组为空 [英] Array of filtered axios results from paginated API is empty

问题描述

在下面的代码中，我的 console.log(response)上得到了一个空数组，但是 getIds 内的 console.log(filterdIds)代码>函数正在显示我想要的数据.我认为我的决心不对.

In my code below I get an empty array on my console.log(response) but the console.log(filterdIds) inside the getIds function is showing my desired data. I think my resolve is not right.

请注意，我一次运行 do..while 进行测试.该API已分页.如果记录来自昨天，它将继续进行；如果不是，则 do..while 停止.

Note that I run do..while once for testing. The API is paged. If the records are from yesterday it will keep going, if not then the do..while is stopped.

有人可以指出我正确的方向吗?

Can somebody point me to the right direction?

const axios = require("axios");

function getToken() {
    // Get the token
}

function getIds(jwt) {
    return new Promise((resolve) => {
        let pageNumber = 1;
        const filterdIds = [];

        const config = {
            //Config stuff
        };

        do {
            axios(config)
                .then((response) => {
                    response.forEach(element => {
                        //Some logic, if true then:
                        filterdIds.push(element.id);
                        console.log(filterdIds);
                    });
                })
                .catch(error => {
                    console.log(error);
                });
        } while (pageNumber != 1)
        resolve(filterdIds);
    });
}


getToken()
    .then(token => {
        return token;
    })
    .then(jwt => {
        return getIds(jwt);
    })
    .then(response => {
        console.log(response);
    })
    .catch(error => {
        console.log(error);
    });

由于 do..while .

推荐答案

基本问题是 resolve(filterdIds); 在请求触发之前同步运行，因此保证为空.

The fundamental problem is that resolve(filterdIds); runs synchronously before the requests fire, so it's guaranteed to be empty.

Promise.all 或 Promise.allSettled 可以帮助您知道需要多少页(或者使用块大小来发出多个请求)-稍后再说).这些方法并行运行.这是一个可运行的概念验证示例:

Promise.all or Promise.allSettled can help if you know how many pages you want up front (or if you're using a chunk size to make multiple requests--more on that later). These methods run in parallel. Here's a runnable proof-of-concept example:

const pages = 10; // some page value you're using to run your loop

axios
  .get("https://httpbin.org") // some initial request like getToken
  .then(response => // response has the token, ignored for simplicity
    Promise.all(
      Array(pages).fill().map((_, i) => // make an array of request promisess
        axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${i + 1}`)
      )
    )
  )
  .then(responses => {
    // perform your filter/reduce on the response data
    const results = responses.flatMap(response =>
      response.data
        .filter(e => e.id % 2 === 0) // some silly filter
        .map(({id, name}) => ({id, name}))
    );
    
    // use the results
    console.log(results);
  })
  .catch(err => console.error(err))
;

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

网络"标签显示并行发生的请求:

The network tab shows the requests happening in parallel:

如果页面数未知，并且您打算一次触发一个请求，直到您的API通知您页面结束，则顺序循环很慢​​，但可以使用.异步/等待更适合此策略:

If the number of pages is unknown and you intend to fire requests one at a time until your API informs you of the end of the pages, a sequential loop is slow but can be used. Async/await is cleaner for this strategy:

(async () => {
  // like getToken; should handle err
  const tokenStub = await axios.get("https://httpbin.org");
  
  const results = [];
  
  // page += 10 to make the snippet run faster; you'd probably use page++
  for (let page = 1;; page += 10) {
    try {
      const url = `https://jsonplaceholder.typicode.com/comments?postId=${page}`;
      const response = await axios.get(url);
      
      // check whatever condition your API sends to tell you no more pages
      if (response.data.length === 0) { 
        break;
      }
      
      for (const comment of response.data) {
        if (comment.id % 2 === 0) { // some silly filter
          const {name, id} = comment;
          results.push({name, id});
        }
      }
    }
    catch (err) { // hit the end of the pages or some other error
      break;
    }
  }
  
  // use the results
  console.log(results);
})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

这是顺序请求瀑布:

如果要增加并行化，可以使用任务队列或分块循环.分块循环将结合两种技术来一次请求 n 条记录，并检查分块中的每个结果是否有终止条件.这是一个简单的示例，它去除了过滤操作，这与异步请求问题是偶然的，并且可以在响应到达后同步完成:

A task queue or chunked loop can be used if you want to increase parallelization. A chunked loop would combine the two techniques to request n records at a time and check each result in the chunk for the termination condition. Here's a simple example that strips out the filtering operation, which is sort of incidental to the asynchronous request issue and can be done synchronously after the responses arrive:

(async () => {  
  const results = [];
  const chunk = 5;
  
  for (let page = 1;; page += chunk) {
    try {
      const responses = await Promise.all(
        Array(chunk).fill().map((_, i) => 
          axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${page + i}`)
        )
      );
      
      for (const response of responses) {      
        for (const comment of response.data) {
          const {name, id} = comment;
          results.push({name, id});
        }
      }
      
      // check end condition
      if (responses.some(e => e.data.length === 0)) { 
        break;
      }
    }
    catch (err) {
      break;
    }
  }
  
  // use the results
  console.log(results);
})();

<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

(上面的图像是100个请求的例外，但是一次可以看到5个块的大小)

(above image is an except of the 100 requests, but the chunk size of 5 at once is visible)

请注意，这些代码段是概念证明，在捕获错误的情况下可能会变得不加区分，请确保捕获所有抛出，等等.将其拆分为子功能时，请确保先 .then 和 await 在调用方中都应有保证-不要尝试将其转换为同步代码.

Note that these snippets are proofs-of-concept and could stand to be less indiscriminate with catching errors, ensure all throws are caught, etc. When breaking it into sub-functions, make sure to .then and await all promises in the caller--don't try to turn it into synchronous code.

另请参见

• 如何从异步方式返回响应和这是什么显式的promise构造反模式，以及如何避免?，警告不要添加新Promise 来帮助解决已经返回promise的代码.

• How do I return the response from an asynchronous call? and Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference which explain why the array is empty.
• What is the explicit promise construction antipattern and how do I avoid it?, which warns against adding a new Promise to help resolve code that already returns promises.

这篇关于来自分页API的已过滤axios结果数组为空的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！