Python的一个干净的，轻量级替代扭曲？ [英] A clean, lightweight alternative to Python's twisted?

问题描述

A（长）前段时间我写了一个网上的蜘蛛，我多线程使并发请求在同一时间发生。那是在我的Python的青春，以前的日子，我知道有关 GIL 和相关的疾苦它创建多线程code（IE，大多数时间东西只是结束连载！）...

A (long) while ago I wrote a web-spider that I multithreaded to enable concurrent requests to occur at the same time. That was in my Python youth, in the days before I knew about the GIL and the associated woes it creates for multithreaded code (IE, most of the time stuff just ends up serialized!)...

我想这返工code，使其更加强大和更好的表现。基本上有两种方法，我可以这样做：我可以2.6+使用新的多模块或我可以去对于某种类型的反应器/基于事件的模型。我宁愿做后期，因为它更简单和更不容易出错。

I'd like to rework this code to make it more robust and perform better. There are basically two ways I could do this: I could use the new multiprocessing module in 2.6+ or I could go for a reactor / event-based model of some sort. I would rather do the later since it's far simpler and less error-prone.

因此​​，问题涉及到什么框架将最适合我的需要。以下是我了解，到目前为止选项列表：

So the question relates to what framework would be best suited to my needs. The following is a list of the options I know about so far:

• 扭曲：Python的反应器框架的鼻祖：看起来复杂，有点臃肿但是。对于一个小任务陡峭的学习曲线。


• Eventlet ：从 lindenlab 的家伙。 Greenlet说的对这些类型的任务面向基础架构。我一看code，虽然这不是太pretty：非PEP8标准，散落着印（人们为什么这样做在一个框架！？这样做），API似乎有点不一致


• PyEv ：在不成熟的，似乎没有要马上使用它的人，虽然它是基于关于libevent的所以它有一个坚实的后台。


• asyncore ：从STDLIB：尤伯杯低级，好像只是参与得到了很多跑腿东西掉在地上。


• 龙卷风：虽然这是一个面向服务器产品，旨在服务器动态网站它配备了的async HTTP客户端和一个简单的ioloop 。看起来可能把工作做好，但不是它的目的是为。
  

• Twisted: The granddaddy of Python reactor frameworks: seems complex and a bit bloated however. Steep learning curve for a small task.
• Eventlet: From the guys at lindenlab. Greenlet based framework that's geared towards these kinds of tasks. I had a look at the code though and it's not too pretty: non-pep8 compliant, scattered with prints (why do people do this in a framework!?), API seems a little inconsistent.
• PyEv: Immature, doesn't seem to be anyone using it right now though it is based on libevent so it's got a solid backend.
• asyncore: From the stdlib: über low-level, seems like a lot of legwork involved just to get something off the ground.
• tornado: Though this is a server oriented product designed to server dynamic websites it does feature an async HTTP client and a simple ioloop. Looks like it could get the job done but not what it was intended for. [edit: doesn't run on Windows unfortunately, which counts it out for me - its a requirement for me to support this lame platform]

有什么我有过一次错过了什么？当然，必须有一个图书馆，在那里，适合一个简化的最佳点异步网络库！

Is there anything I have missed at all? Surely there must be a library out there that fits the sweet-spot of a simplified async networking library!

推荐答案

我喜欢的同意的Python模块，要么依赖于Stackless Python才能微线程或Greenlets用于轻型线程。所有阻塞网络I / O是透明异步通过一个单一的的libevent 循环做，所以它应该是几乎与真正的异步服务器的高效性能。

I liked the concurrence Python module which relies on either Stackless Python microthreads or Greenlets for light-weight threading. All blocking network I/O is transparently made asynchronous through a single libevent loop, so it should be nearly as efficient as an real asynchronous server.

我想这是类似于这种方式Eventlet。

I suppose it's similar to Eventlet in this way.

缺点是，它的API是Python的插槽 / 线程模块完全不同的;你需要重写应用程序的公平位（或写兼容性垫片层）

The downside is that its API is quite different from Python's sockets/threading modules; you need to rewrite a fair bit of your application (or write a compatibility shim layer)

编辑：似乎有也热电联产，这是类似，但使用Python 2.5的的增强发电机的其协同程序，而不是Greenlets。这使得它比同意和其他替代更加便于携带。网络I / O直接与epoll的/ kqueue的/ IOCP完成。

It seems that there's also cogen, which is similar, but uses Python 2.5's enhanced generators for its coroutines, instead of Greenlets. This makes it more portable than concurrence and other alternatives. Network I/O is done directly with epoll/kqueue/iocp.

这篇关于Python的一个干净的，轻量级替代扭曲？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！