如何在Python ssl套接字缓冲中使用select? [英] How to use select with Python ssl socket buffering?
问题描述
我的问题类似于 python-select.select()如何工作? .但是，那里的解决方案对我不起作用，因为我没有打开()文件.相反，它是一个套接字.我在文档中找不到任何将其设置为不缓冲的方法.
My problem is similar to python - How select.select() works? . However, the solution there doesn't work for me, because I'm not open()ing my file. Instead, it's a socket. I couldn't find any way to set it to be unbuffered in the documentation.
我有一个glib mainloop(使用select)，在其中注册了要读取的套接字.因为socket.recv()要求我指定接收缓冲区的大小，所以读取少于读取的套接字的字节并不稀奇.只要内核对它们进行缓冲，就可以了. select仍会将套接字标记为准备读取".但显然Python也有一个缓冲区.对于大文件，在数据流的末尾，recv()会读取其中的一部分，其余部分将由Python缓冲，并在我的套接字上不再选择触发器，直到发送新数据.在这一点上，丢失"的数据在新数据之前被接收；没有数据丢失.
I have a glib mainloop (which uses select), where I registered the socket for reading. Because socket.recv() requires me to specify a receive buffer size, it is not unusual to read fewer bytes than the socket read. As long as the kernel buffers them, that is fine; select will still mark the socket as "ready for reading". But apparently Python has a buffer as well. With large files, near the end of the data stream, recv() will read a part of it, the rest will be buffered by Python and select no longer triggers on my socket, until new data is sent. At that point, the "missing" data is received before the new data; no data is lost.
我的问题是:我该如何解决?有没有一种方法可以在套接字上禁用Python的缓冲区?如果不是，是否有办法检查缓冲区是否为空，所以我可以确保在回调之前不要从回调中返回?
My question is: how do I solve this? Is there a way to disable Python's buffer on the socket? If not, is there a way to check if the buffer is empty, so I can make sure I don't return from my callback until it is?
如注释中所述，Python不会向套接字添加额外的缓冲区，因此这可能不是问题.我无法为该问题创建一个最小的示例.但是，这似乎与使用ssl套接字有关.我忘记了我使用加密连接.禁用加密似乎可以解决此问题，但是我不接受.因此，上面的问题仍然存在，请注意，缓冲区可能是在ssl模块中实现的.
As noted in the comment, Python doesn't add an extra buffer to sockets, so this could not be the problem. I was unable to create a minimal example for the problem. However, it seems that it may be related to using ssl sockets. I had forgotten that I used an encrypted connection; disabling the encryption seems to solve this issue, but is not acceptable to me. So the above question remains, with the note that the buffers are probably implemented in the ssl module.
显示问题的示例代码:
#!/usr/bin/python
import glib
import socket
import ssl
def cb (fd, cond):
print ('data: %s' % repr (s.read (1)))
return True
s = ssl.wrap_socket (socket.create_connection (('localhost', 1234)))
glib.io_add_watch (s.fileno (), glib.IO_IN, cb)
glib.MainLoop ().run ()
然后使用以下命令运行服务器
Then run a server with
openssl s_server -accept 1234 -key file.key -cert file.crt
运行python程序将建立连接.发送多于一个字节的数据将使程序仅打印第一个字节.发送更多字节时，将先读取其余的块，然后读取第一个新字节，然后再次等待.这很容易理解:只要ssl缓冲区中有数据，就不会从内核缓冲区中读取新字节，因此select继续报告它.
Running the python program will establish the connection. Sending more than one byte of data will make the program print only the first byte; when sending more bytes, the remaining chunks are read first, then the first new byte, then it waits again. This is easy to understand: as long as there is data in the ssl buffer, the new byte is not read from the kernel buffer, so select continues to report it.
推荐答案
在查看ssl源代码时，我发现了一个未记录的函数，该函数可以满足我的需要:pending().可以这样使用:
Looking into the ssl source, I found an undocumented function which does what I want: pending(). It can be used like so:
#!/usr/bin/python
import glib
import socket
import ssl
def cb(fd, cond):
print('data: %s' % repr(s.read(1)))
while(s.pending()):
print('more data: %s' % repr(s.read(1)))
return True
s = ssl.wrap_socket (socket.create_connection(('localhost', 1234)))
glib.io_add_watch(s.fileno(), glib.IO_IN, cb)
glib.MainLoop().run()
这可以解决问题.
这篇关于如何在Python ssl套接字缓冲中使用select?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
