Python 套接字缓冲 [英] Python sockets buffering

问题描述

假设我想使用标准的 socket 模块从套接字读取一行:

Let's say I want to read a line from a socket, using the standard socket module:

def read_line(s):
    ret = ''

    while True:
        c = s.recv(1)

        if c == '
' or c == '':
            break
        else:
            ret += c

    return ret

s.recv(1) 到底发生了什么?每次都会发出系统调用吗?无论如何，我想我应该添加一些缓冲:

What exactly happens in s.recv(1)? Will it issue a system call each time? I guess I should add some buffering, anyway:

为了与硬件和网络现实最佳匹配，bufsize 的值应该是相对较小的 2 的幂，例如 4096.

For best match with hardware and network realities, the value of bufsize should be a relatively small power of 2, for example, 4096.

http://docs.python.org/library/socket.html#socket.socket.recv

但是编写高效且线程安全的缓冲似乎并不容易.如果我使用 file.readline() 会怎样?

But it doesn't seem easy to write efficient and thread-safe buffering. What if I use file.readline()?

# does this work well, is it efficiently buffered?
s.makefile().readline()

推荐答案

recv()调用直接通过调用C库函数处理.

The recv() call is handled directly by calling the C library function.

它会阻塞等待套接字有数据.实际上它只会让 recv() 系统调用阻塞.

It will block waiting for the socket to have data. In reality it will just let the recv() system call block.

file.readline() 是一种高效的缓冲实现.它不是线程安全的，因为它假定它是唯一一个读取文件的人.(例如通过缓冲即将到来的输入.)

file.readline() is an efficient buffered implementation. It is not threadsafe, because it presumes it's the only one reading the file. (For example by buffering upcoming input.)

如果您使用的是文件对象，每次使用正参数调用read() 时，底层代码只会recv() 请求的数据量, 除非它已经缓冲了.

If you are using the file object, every time read() is called with a positive argument, the underlying code will recv() only the amount of data requested, unless it's already buffered.

如果:

• 你调用了 readline()，它读取一个完整的缓冲区

• you had called readline(), which reads a full buffer

行的结尾在缓冲区的结尾之前

the end of the line was before the end of the buffer

因此将数据留在缓冲区中.否则缓冲区通常不会溢出.

Thus leaving data in the buffer. Otherwise the buffer is generally not overfilled.

问题的目的不明确.如果需要在读取前查看数据是否可用，可以select() 或使用s.setblocking(False) 将套接字设置为非阻塞模式.然后，如果没有等待数据，读取将返回空，而不是阻塞.

The goal of the question is not clear. if you need to see if data is available before reading, you can select() or set the socket to nonblocking mode with s.setblocking(False). Then, reads will return empty, rather than blocking, if there is no waiting data.

您是在读取一个文件或多个线程的套接字吗?我会让一个工作人员读取套接字并将接收到的项目送入队列以供其他线程处理.

Are you reading one file or socket with multiple threads? I would put a single worker on reading the socket and feeding received items into a queue for handling by other threads.

建议咨询Python 套接字模块源 和 进行系统调用的 C 源.

这篇关于Python 套接字缓冲的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！