由于每秒上限而限制和排队API请求 [英] Throttle and queue up API requests due to per second cap

问题描述

我正在使用 mikeal/request 进行API调用.我最常使用的API之一(Shopify API).最近推出了新的通话限制 ，我看到如下错误:

I'm use mikeal/request to make API calls. One of the API's I use most frequently (the Shopify API). Recently put out a new call limit, I'm seeing errors like:

Exceeded 6.0 calls per second for api client. Slow your requests or contact support for higher limits.

我已经进行了升级，但是无论获得多少带宽，我都必须考虑这一点.对Shopify API的大部分请求都在 async.map()函数中，该函数会循环异步请求，然后收集尸体.

I've already gotten an upgrade, but regardless of how much bandwidth I get I have to account for this. A large majority of the requests to the Shopify API are within async.map() functions, which loop asynchronous requests, and gather the bodies.

我正在寻找任何帮助，也许是一个已经存在的库，该库将包裹在请求模块周围，并实际上阻止，睡眠，调节，分配，管理所有异步发出的并发请求并将它们限制为一次说6个请求.如果不存在这样的项目，我没有任何问题.我只是不知道如何处理这种情况，我希望有某种标准.

I'm looking for any help, perhaps a library that already exists, that would wrap around the request module and actually block, sleep, throttle, allocate, manage, the many simultaneous requests that are firing off asynchronously and limit them to say 6 requests at a time. I have no problem with working on such a project if it doesn't exist. I just don't know how to handle this kind of situation, and I'm hoping for some kind of standard.

我通过 mikeal/request 制作了一张票.

推荐答案

我在使用各种AP​​I时遇到了同样的问题. AWS也以节流而闻名.

I've run into the same issue with various APIs. AWS is famous for throttling as well.

可以使用两种方法.您提到了async.map()函数.您是否尝试过 async.queue()?队列方法应允许您设置固定限制(例如6)，超过该限制的任何内容都将放入队列中.

A couple of approaches can be used. You mentioned async.map() function. Have you tried async.queue()? The queue method should allow you to set a solid limit (like 6) and anything over that amount will be placed in the queue.

另一个有用的工具是 oibackoff .如果从服务器返回错误并再次尝试，该库将允许您回退请求.

Another helpful tool is oibackoff. That library will allow you to backoff your request if you get an error back from the server and try again.

包装两个库以确保两个库都被覆盖可能是有用的:async.queue以确保不超出限制，而oibackoff以确保在遇到请求时再次尝试获取请求服务器告诉您有错误.

It can be useful to wrap the two libraries to make sure both your bases are covered: async.queue to ensure you don't go over the limit, and oibackoff to ensure you get another shot at getting your request in if the server tells you there was an error.

这篇关于由于每秒上限而限制和排队API请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！