Twisted中Select/Poll与Epoll反应器的警告 [英] Caveats of select/poll vs. epoll reactors in Twisted

问题描述

我阅读和经历过的所有内容(基于Tornado的应用程序)使我相信ePoll是基于Select和Poll的网络的自然替代品，尤其是Twisted.这让我感到偏执，对于一个更好的技术或方法学来说，这是很罕见的，而没有价格.

Everything I've read and experienced ( Tornado based apps ) leads me to believe that ePoll is a natural replacement for Select and Poll based networking, especially with Twisted. Which makes me paranoid, its pretty rare for a better technique or methodology not to come with a price.

通过阅读epoll与替代方案之间的几十个比较结果，可以看出epoll显然是速度和可伸缩性的拥护者，特别是它以线性方式扩展，这太棒了.也就是说，关于处理器和内存利用率，epoll仍然是冠军吗?

Reading a couple dozen comparisons between epoll and alternatives shows that epoll is clearly the champion for speed and scalability, specifically that it scales in a linear fashion which is fantastic. That said, what about processor and memory utilization, is epoll still the champ?

推荐答案

对于数量很少的套接字(当然，取决于您的硬件，但我们谈论的是10个或更少的数量)，请选择可以在内存使用率和运行速度上击败epoll.当然，对于这么少的套接字，这两种机制都是如此之快，以至于您在大多数情况下都不关心这种差异.

For very small numbers of sockets (varies depending on your hardware, of course, but we're talking about something on the order of 10 or fewer), select can beat epoll in memory usage and runtime speed. Of course, for such small numbers of sockets, both mechanisms are so fast that you don't really care about this difference in the vast majority of cases.

不过，请澄清一下.选择和epoll都线性缩放.但是，一个很大的区别是，面向用户空间的API具有基于不同事物的复杂性. select调用的开销与传递给它的编号最高的文件描述符的值大致相同.如果选择单一fd(100)，则价格大约是选择单一fd(50)的两倍.在最高的fd以下添加更多的fd并非完全免费，因此实际上比这要复杂一些.对于大多数实现而言，它是一个很好的第一近似值.

One clarification, though. Both select and epoll scale linearly. A big difference, though, is that the userspace-facing APIs have complexities that are based on different things. The cost of a select call goes roughly with the value of the highest numbered file descriptor you pass it. If you select on a single fd, 100, then that's roughly twice as expensive as selecting on a single fd, 50. Adding more fds below the highest isn't quite free, so it's a little more complicated than this in practice, but this is a good first approximation for most implementations.

epoll的成本接近实际具有事件的文件描述符的数量.如果您要监视200个文件描述符，但是其中只有100个文件描述符具有事件，那么(非常粗略地)您只需为这100个活动文件描述符付费.这是epoll相对于select倾向于提供其主要优势之一的地方.如果您有一千个客户大部分都是闲置的，那么当您使用select时，您仍然需要为全部一千个客户付费.但是，使用epoll，就好像您只有几个-您只需为在任何给定时间处于活动状态的活动付费.

The cost of epoll is closer to the number of file descriptors that actually have events on them. If you're monitoring 200 file descriptors, but only 100 of them have events on them, then you're (very roughly) only paying for those 100 active file descriptors. This is where epoll tends to offer one of its major advantages over select. If you have a thousand clients that are mostly idle, then when you use select you're still paying for all one thousand of them. However, with epoll, it's like you've only got a few - you're only paying for the ones that are active at any given time.

所有这些都意味着epoll将减少大多数工作负载的CPU使用率.就内存使用而言，这有点麻烦. select确实以高度紧凑的方式表示了所有必要的信息(每个文件描述符一位). FD_SETSIZE(通常为1024)限制了您可以与select一起使用的文件描述符的数量，这意味着您可以为可与select一起使用的三个fd集合中的每一个花费不超过128个字节(读，写，例外).与那些最大384个字节相比，epoll有点像猪.每个文件描述符都由一个多字节结构表示.但是，从绝对意义上讲，它仍然不会占用太多内存.您可以在几十个千字节中表示大量的文件描述符(我认为每1000个文件描述符大约20k).而且，如果您只想监视一个文件描述符，但是它的值恰好是1024，那么如果您只想监视一个文件描述符，则必须用select花费所有384个字节，而使用epoll则只需花费20个字节.尽管如此，所有这些数字都很小，所以并没有太大的区别.

All this means that epoll will lead to less CPU usage for most workloads. As far as memory usage goes, it's a bit of a toss up. select does manage to represent all the necessary information in a highly compact way (one bit per file descriptor). And the FD_SETSIZE (typically 1024) limitation on how many file descriptors you can use with select means that you'll never spend more than 128 bytes for each of the three fd sets you can use with select (read, write, exception). Compared to those 384 bytes max, epoll is sort of a pig. Each file descriptor is represented by a multi-byte structure. However, in absolute terms, it's still not going to use much memory. You can represent a huge number of file descriptors in a few dozen kilobytes (roughly 20k per 1000 file descriptors, I think). And you can also throw in the fact that you have to spend all 384 of those bytes with select if you only want to monitor one file descriptor but its value happens to be 1024, wheras with epoll you'd only spend 20 bytes. Still, all these numbers are pretty small, so it doesn't make much difference.

epoll还有其他好处，您可能已经知道了，它不仅限于FD_SETSIZE文件描述符.您可以使用它来监视尽可能多的文件描述符.而且，如果您只有一个文件描述符，但是其值大于FD_SETSIZE，则epoll也可以使用该文件描述符，但是select不能.

And there's also that other benefit of epoll, which perhaps you're already aware of, that it is not limited to FD_SETSIZE file descriptors. You can use it to monitor as many file descriptors as you have. And if you only have one file descriptor, but its value is greater than FD_SETSIZE, epoll works with that too, but select does not.

随机地，我最近还发现与select或poll相比，epoll的一个小缺点.尽管这三个API都不支持普通文件(即文件系统上的文件)，但select和poll缺乏这种支持，因为它们将此类描述符报告为始终可读且始终可写.这使它们不适用于任何有意义的非阻塞文件系统I/O，使用select或poll并且碰巧遇到文件系统中文件描述符的程序将至少继续运行(或者，如果失败，可能不是因为select或poll)，尽管它可能不是具有最佳性能.

Randomly, I've also recently discovered one slight drawback to epoll as compared to select or poll. While none of these three APIs supports normal files (ie, files on a file system), select and poll present this lack of support as reporting such descriptors as always readable and always writeable. This makes them unsuitable for any meaningful kind of non-blocking filesystem I/O, a program which uses select or poll and happens to encounter a file descriptor from the filesystem will at least continue to operate (or if it fails, it won't be because of select or poll), albeit it perhaps not with the best performance.

另一方面，当要求epoll监视此类文件描述符时，它会快速失败并显示错误(显然是EPERM).严格来说，这几乎是不正确的.它只是以明确的方式表明其缺乏支持.通常，我会为显式的故障情况表示赞赏，但是这种情况是无证的(据我所知)，并且会导致应用程序完全损坏，而不是仅以可能降低性能的方式运行.

On the other hand, epoll will fail fast with an error (EPERM, apparently) when asked to monitor such a file descriptor. Strictly speaking, this is hardly incorrect. It's merely signalling its lack of support in an explicit way. Normally I would applaud explicit failure conditions, but this one is undocumented (as far as I can tell) and results in a completely broken application, rather than one which merely operates with potentially degraded performance.

在实践中，我唯一看到的地方是与stdio进行交互时.用户可以将stdin或stdout从/重定向到普通文件.以前的stdin和stdout会是一个管道-epoll可以很好地支持它-然后它变成一个普通文件，epoll大声失败，从而破坏了应用程序.

In practice, the only place I've seen this come up is when interacting with stdio. A user might redirect stdin or stdout from/to a normal file. Whereas previously stdin and stdout would have been a pipe -- supported by epoll just fine -- it then becomes a normal file and epoll fails loudly, breaking the application.

这篇关于Twisted中Select/Poll与Epoll反应器的警告的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！