如何提高gevent和龙卷风组合的性能? [英] How to improve the performance of the combination of gevent and tornado?
问题描述
我正在尝试将gevent用作wsgi服务器，并使用龙卷风WSGIApplication处理请求.这是代码
I am trying to use gevent as wsgi server, and use tornado WSGIApplication to process requests. Here's the code
#!/usr/bin/env python
# coding=utf-8
import gevent
from gevent import monkey
monkey.patch_all(thread=False)
from gevent.pywsgi import WSGIServer
from tornado.wsgi import WSGIApplication
import tornado.web
import tornado.wsgi
import requests
class MainHandler(tornado.web.RequestHandler):
def get(self):
requests.get('http://google.com')
self.write('hello')
handlers = [
(r'/', MainHandler)
]
if __name__ == '__main__':
application = WSGIApplication(handlers)
server = WSGIServer(('', 9010), application)
server.serve_forever()
然后我使用apache基准测试来测试性能.测试命令是
And I use apache benchmark to test the performance. Test command is
ab -n 1000 -c 100 http://127.0.0.1:9010/
这导致每秒100req，太慢了.在上面的代码中，我只是将http请求发送到另一端，我认为在这种情况下，gevent会在阻塞时切换到其他greenlet，这对性能几乎没有影响，但是gevent的性能从每秒1600req降低到100每秒请求数，我不知道为什么.
This resulting 100req per second, it is too slow. In the above code, I just put a http request to other side, I think in this situation, gevent will switch to other greenlet when block, and it should has little influence on the performance, but gevent's performance decrease from 1600req per second to 100 req per second, I can't figure out why.
有人可以解释吗?
推荐答案
您好，问题在于您没有生成实际的greenlet，并且tornado.web.ascynhronous装饰器不支持WSGI服务器.
Hi you're issue is that you aren't spawning an actual greenlet and that the tornado.web.ascynhronous decorator does nos support WSGI servers.
但是主要逻辑起作用，并且我能够使它与HTTP服务器一起工作(我不知道您是否绑定到WSGI服务器，但我想不是，因为您也可以反向代理)
But the main logic works and I was able to get it working with a HTTP server (I don't know if you're tied to a WSGI server, but I guess not since you can just reverse proxy just as well).
我发现很多人都想将gevent与龙卷风一起使用，包括我在内(我们在 FriendCode 中使用了tornado和gevent) ，所以我这样写:
I find a lot of people wanting to use gevent with tornado, me included (we use tornado and gevent at FriendCode), so I wrote this:
# Gevent monkeypath
from gevent import monkey
monkey.patch_all()
# Gevent imports
import gevent
# Python immports
import functools
# Tornado imports
import tornado.ioloop
import tornado.web
import tornado.httpserver
# Request imports
import requests
# Asynchronous gevent decorator
def gasync(func):
@tornado.web.asynchronous
@functools.wraps(func)
def f(self, *args, **kwargs):
#self._auto_finish = False
return gevent.spawn(func, self, *args, **kwargs)
return f
# Constants
URL_TO_FETCH = 'http://google.co.uk/'
# Global
I = 0
class MainHandler(tornado.web.RequestHandler):
@gasync
def get(self):
global I
r = requests.get(URL_TO_FETCH)
I += 1
print('Got page %d (length=%d)' % (I, len(r.content)))
self.write("Done")
self.finish()
# Our URL Mappings
handlers = [
(r"/", MainHandler),
]
def main():
# Setup app and HTTP server
application = tornado.web.Application(handlers)
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(9998)
# Start ioloop
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
在此示例中，有两个关键部分，您正确的猴子修补部分，然后是我编写的 gasync 装饰器，它所做的全部就是将方法设置为异步(按照龙卷风的说法) ，这意味着该方法必须自行调用self.finish()才能将响应发送给客户端，因为龙卷风会在请求同步时自动调用它，但这不是异步所需要的.
In this sample there are two key parts, the monkey patching part that you got right and then the gasync decorator that I wrote, all it does is set a method to being asynchronous (by tornado's terms, which means that the method has to call self.finish() by itself to send the response to the client, because tornado calls it automatically when the request is synchronous, but it's not what you want in async).
我希望对您有所帮助，我在这里用:
I hope that helps, the code works fine here I've tested it with :
$ ab -n 100 -c 100 http://localhost:9998/
哪个给:
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: TornadoServer/2.3
Server Hostname: localhost
Server Port: 9998
Document Path: /
Document Length: 4 bytes
Concurrency Level: 100
Time taken for tests: 0.754 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 15900 bytes
HTML transferred: 400 bytes
Requests per second: 132.67 [#/sec] (mean)
Time per request: 753.773 [ms] (mean)
Time per request: 7.538 [ms] (mean, across all concurrent requests)
Transfer rate: 20.60 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 4 0.8 4 5
Processing: 379 572 104.4 593 748
Waiting: 379 572 104.4 593 748
Total: 383 576 104.3 596 752
Percentage of the requests served within a certain time (ms)
50% 596
66% 640
75% 672
80% 679
90% 707
95% 722
98% 735
99% 752
100% 752 (longest request)
您可以看到总时间大约等于最长请求的时间，请记住，当异步时:
As you can see the total time there is roughly equal to the time of the longest request, remember that when async :
total_time = max(all_individual_times) + n*some_overhead
其中 n 是请求数，而 some_overhead 是恒定的开销.
Where n is the number of requests, and some_overhead a constant overhead.
希望有帮助:)
这篇关于如何提高gevent和龙卷风组合的性能?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
