用安稳的方式批量处理API请求 [英] Batching API requests with flask-restful

问题描述

$ b

我正在构建一个REST API，其中有一点我想要启用的是批量请求资源的能力，类似于Facebook Graph API的工作方式。 $ b

  curl \ 
 -F'access_token = ...'\ 
 -F'batch = [{method：GET， relative_url：me}，{method：GET，ative_url：me / friends？limit = 50}]'\ 
 https://graph.facebook.com 
  

然后返回一个数组，每个请求都使用状态码和结果解析：

  [
 {code：200，
headers：[
 {name类型，
value：text / javascript; charset = UTF-8} 
]，
body：{\id \：\ ... \}}，
 {code：200，
headers：[
 {name：Content-Type，
value ：text / javascript; charset = UTF-8} 
]，
body：{\data \：[{...}]}} 
] 
  

我已经能够通过循环请求和调用urlopen对我自己的应用程序。这似乎效率很低，我不得不认为有一个更好的办法。有一个更简单的和/或更好的方法来请求处理程序内的请求对我自己的应用程序？由于您需要返回标题，因此我建议您将这些批处理的请求发送给自己（即将请求发送到<$>

c $ c> localhost ），那样的话响应将与你在进行单个调用时得到的响应一致。

你的API收到一个批处理请求，你将需要至少有一个免费的工作人员来处理这些间接的请求，而第一个工人阻止和等待。所以你需要至少有两名工人。即使如此，如果两个批处理请求同时到达并且带走两名工作人员，则可能会导致死锁。所以在现实中，你需要拥有尽可能多的工人，比如你希望同时收到的批量请求，再加上至少一个来处理间接请求。

从另一个角度看，您将需要并行运行尽可能多的这些间接请求，因为如果它们最终运行一个接一个，则使用批处理请求的好处将会丢失。所以你也需要有足够数量的工人来允许并行。

诚然，我不认为这是一个很好的特性。在大多数客户端语言中，并行执行多个请求相当容易，所以您不需要提供这是一个服务器端功能。特别容易，如果你正在使用Javascript，但也很容易在Python，Ruby等。

I'm building a REST API with flask-restful and one thing I'd like to enable is the ability to batch request resources, similar to how the Facebook Graph API works:

curl \
    -F 'access_token=…' \
    -F 'batch=[{"method":"GET", "relative_url":"me"},{"method":"GET", "relative_url":"me/friends?limit=50"}]' \
    https://graph.facebook.com

Which then returns an array with each request resolved with its status code and result:

[
    { "code": 200, 
      "headers":[
          { "name": "Content-Type", 
            "value": "text/javascript; charset=UTF-8" }
      ],
      "body": "{\"id\":\"…\"}"},
    { "code": 200,
      "headers":[
          { "name":"Content-Type", 
            "value":"text/javascript; charset=UTF-8"}
      ],
      "body":"{\"data\": [{…}]}}
]

I've been able to replicate this in flask-restful by simply looping over the requests and calling urlopen against my own app. This seems really inefficient and I have to think there's a better way. Is there a simpler and/or better way to make the requests against my own application from within a request handler?

解决方案

Since you need to return headers my recommendation is that you send these batched requests into yourself (i.e. send the requests to localhost), that way the responses will be consistent with those that you would get when making individual calls.

Consider that when your API receives a batch request you will need at least one free worker to take on these indirect requests while the first worker blocks and waits. So you will need to have at least two workers. Even with this you risk a deadlock, if two batch requests arrive at the same time and take your two workers. So in reality you need to have as many workers as batch requests you expect to receive concurrently plus at least one more to handle the indirect requests.

Looking at this from another side, you will want to run as many of these indirect requests in parallel, because if they end up running one after another the benefit of using batch requests is lost. So you also need to have sufficient number of workers to allow for parallelism.

In all honesty I don't see this as a great feature. In most client-side languages it is fairly easy to execute several requests in parallel, so you don't need to provide this is a server-side feature. Specially easy if you are using Javascript, but also easy in Python, Ruby, etc.

这篇关于用安稳的方式批量处理API请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！