向一个 API 发出多个请求，每分钟只能处理 20 个请求 [英] Make several requests to an API that can only handle 20 request a minute

问题描述

I've got a method that returns a promise and internally that method makes a call to an API which can only have 20 requests every minute. The problem is that I have a large array of objects (around 300) and I would like to make a call to the API for each one of them.

At the moment I have the following code:

    const bigArray = [.....];

    Promise.all(bigArray.map(apiFetch)).then((data) => {
      ...
    });

But it doesnt handle the timing constraint. I was hoping I could use something like _.chunk and _.debounce from lodash but I can't wrap my mind around it. Could anyone help me out ?

解决方案

You could send 1 block of 20 requests every minute or space them out 1 request every 3 seconds (latter probably preferred by the API owners).

function rateLimitedRequests(array, chunkSize) {
  var delay = 3000 * chunkSize;
  var remaining = array.length;
  var promises = [];
  var addPromises = function(newPromises) {
    Array.prototype.push.apply(promises, newPromises);
    if (remaining -= newPromises.length == 0) {
      Promise.all(promises).then((data) => {
        ... // do your thing
      });
    }
  };
  (function request() {
    addPromises(array.splice(0, chunkSize).map(apiFetch));
    if (array.length) {
      setTimeout(request, delay);
    }
  })();
}

To call 1 every 3 seconds:

rateLimitedRequests(bigArray, 1);

Or 20 every minute:

rateLimitedRequests(bigArray, 20);

If you prefer to use _.chunk and _.debounce¹ _.throttle:

function rateLimitedRequests(array, chunkSize) {
  var delay = 3000 * chunkSize;
  var remaining = array.length;
  var promises = [];
  var addPromises = function(newPromises) {
    Array.prototype.push.apply(promises, newPromises);
    if (remaining -= newPromises.length == 0) {
      Promise.all(promises).then((data) => {
        ... // do your thing
      });
    }
  };
  var chunks = _.chunk(array, chunkSize);  
  var throttledFn = _.throttle(function() {
    addPromises(chunks.pop().map(apiFetch));
  }, delay, {leading: true});
  for (var i = 0; i < chunks.length; i++) {
    throttledFn();
  }
}

¹You probably want _.throttle since it executes each function call after a delay whereas _.debounce groups multiple calls into one call. See this article linked from the docs

Debounce: Think of it as "grouping multiple events in one". Imagine that you go home, enter in the elevator, doors are closing... and suddenly your neighbor appears in the hall and tries to jump on the elevator. Be polite! and open the doors for him: you are debouncing the elevator departure. Consider that the same situation can happen again with a third person, and so on... probably delaying the departure several minutes.

Throttle: Think of it as a valve, it regulates the flow of the executions. We can determine the maximum number of times a function can be called in certain time. So in the elevator analogy.. you are polite enough to let people in for 10 secs, but once that delay passes, you must go!

这篇关于向一个 API 发出多个请求，每分钟只能处理 20 个请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！