Python请求模块GET / POST +各种REST客户端的请求花费的时间比curl长 [英] Python requests module GET/POST + various REST clients' requests taking longer than curl

问题描述

基本上，我有一个url，我打算得到一些XML数据。端点我不能透露，但执行：

  curl -vhttp://my-url.com/some/endpoint 
  

几乎立即返回200 OK 和内容 p>

使用Kenneth Reitz的请求模块，我有一个POST请求和一个GET请求，都需要30秒才能返回内容。

如果我这样使用：

 从timeit导入定时器
 
t =定时器（lambda：requests.get（myurl）.content）
 print t.timeit（number = 1）
 30.2136261463 
  

每次平均需要30.2秒。与我的POST请求相同。如果我不要求内容和status_code响应，我得到相同的情况，除非我通过 stream = True ，我得到快速响应，但不是内容。 p>

我的困惑是在curl命令内...我得到的响应和内容在10ms以下。我尝试伪造用户代理在我的python测试，尝试传递许多参数到get（）函数等。在curl和python请求之间必须有一些主要的区别，我不知道的请求。我是新手，所以如果我缺少一些明显的东西，我道歉。

我还想提到我已经尝试过多个机器，多个版本的卷曲，python，甚至尝试一些REST客户端像Postman等。只有卷曲执行闪电快 - 击中相同的端点在每种情况下BTW。我理解其中一个选项是在我的测试中执行子进程调用curl，但是...这是一个好主意？

编辑：
关心内容。我知道我可以快速获得响应代码（标题）。

提前感谢，

Tihomir。

UPDATE：

我现在在测试中使用pycurl2，只是一个解决方法，因为我希望我可以使用python请求的一切。

解决方案

由于这个问题根本没有产生任何兴趣，我会去接受我自己的解决方案解决方案 - 这涉及使用pycurl2而不是请求有问题的请求。

只有2个都很慢，这样做固定了我的问题，但它不是我希望的解决方案。

注意：我不是说，任何方式，请求是慢或坏。这似乎是一个问题与gzip压缩和GlassFish服务Gzipped数据与一个错误的长度。我只是想知道为什么它不影响curl / wget。

Basically, I have a url that I'm hitting to get some XML data. The endpoint I cannot disclose, but doing:

curl -v "http://my-url.com/some/endpoint"

returns a 200 OK and the content pretty much instantly.

Using the requests module by Kenneth Reitz, I have both a POST request and a GET request that both take 30 seconds to return content.

If I use it this way:

from timeit import Timer

t = Timer(lambda: requests.get(myurl).content)
print t.timeit(number=1)
30.2136261463

it takes 30.2 sec on average each time. Same with my POST request. If I don't ask for content and just the status_code response, I get the same situation, unless if I pass the stream=True, where I get the response quickly, but not the content.

My confusion is within the curl command... I get both the response and content in under 10ms. I tried faking the user-agent in my python test, tried passing numerous arguments to the get() function etc. There must be some major difference between how curl and python-requests do requests that I am not aware of. I am a newbie, so I do apologise if I am missing something obvious.

I would also like to mention that I have tried multiple machines for this, multiple version of curl, python and even tried some REST clients like Postman etc. Only curl performs lightning fast - hitting the same endpoint in every case BTW. I understand one of the options is to do a subprocess call to curl within my test, but... Is that a good idea?

EDIT: I care about the content. I am aware I can get the response code quickly (headers).

Thanks in advance,

Tihomir.

UPDATE:

I am now using pycurl2 in my test, so this is just a workaround as I was hoping I could use python-requests for everything. Still curious as to why is curl so much faster.

解决方案

Since this question is not generating any interest at all, I am going to accept my own workaround solution - which involves using pycurl2 instead of requests for the problematic requests.

Only 2 of all of them are slow, and doing this fixed my issue, but it's not a solution I was hoping for.

NOTE: I am not saying in any way that requests is slow or bad. This seemed to be an issue with gzip compression and GlassFish serving gzipped data with a buggy length. I just wanted to know why it's not affecting curl/wget.

这篇关于Python请求模块GET / POST +各种REST客户端的请求花费的时间比curl长的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！