如何启用请求异步模式? [英] how enable requests async mode?
本文介绍了如何启用请求异步模式?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
对于此代码:
import sys
import gevent
from gevent import monkey
monkey.patch_all()
import requests
import urllib2
def worker(url, use_urllib2=False):
if use_urllib2:
content = urllib2.urlopen(url).read().lower()
else:
content = requests.get(url, prefetch=True).content.lower()
title = content.split('<title>')[1].split('</title>')[0].strip()
urls = ['http://www.mail.ru']*5
def by_requests():
jobs = [gevent.spawn(worker, url) for url in urls]
gevent.joinall(jobs)
def by_urllib2():
jobs = [gevent.spawn(worker, url, True) for url in urls]
gevent.joinall(jobs)
if __name__=='__main__':
from timeit import Timer
t = Timer(stmt="by_requests()", setup="from __main__ import by_requests")
print 'by requests: %s seconds'%t.timeit(number=3)
t = Timer(stmt="by_urllib2()", setup="from __main__ import by_urllib2")
print 'by urllib2: %s seconds'%t.timeit(number=3)
sys.exit(0)
这个结果:
by requests: 18.3397213892 seconds
by urllib2: 2.48605842363 seconds
在嗅探器中看起来是这样的:
in sniffer it looks this:
描述:前 5 个请求由 requests 库发送,接下来的 5 个请求由 urllib2 库发送.红色 - 工作冻结的时间,黑暗 - 接收数据时......wtf?!
description: first 5 requests are sended by requests library, next 5 requests are sended by urllib2 library. red - is time when work was freeze, dark - when data receiving... wtf?!
如果套接字库打补丁并且库必须相同地工作怎么可能?如何使用没有 requests.async 的请求进行异步工作?
推荐答案
抱歉 Kenneth Reitz.他的图书馆很棒.
Sorry Kenneth Reitz. His library is wonderful.
我很笨.我需要像这样为 httplib 选择猴子补丁:
I am stupid. I need select monkey patch for httplib like this:
gevent.monkey.patch_all(httplib=True)
因为 httplib 的补丁默认是禁用的.
Because patch for httplib is disabled by default.
这篇关于如何启用请求异步模式?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文