Python解释器会阻止多线程DNS请求吗? [英] Python Interpreter blocks Multithreaded DNS requests?

问题描述

我只是用python和线程玩了一点，甚至意识到即使在多线程脚本中，DNS请求也被阻止了.考虑以下脚本:

I just played around a little bit with python and threads, and realized even in a multithreaded script, DNS requests are blocking. Consider the following script:

从线程导入线程 导入套接字

from threading import Thread import socket

class Connection(Thread):
    def __init__(self, name, url):
        Thread.__init__(self)
        self._url = url
        self._name = name

    def run(self):
        print "Connecting...", self._name
        try:
            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            s.setblocking(0)
            s.connect((self._url, 80))
        except socket.gaierror:
            pass #not interested in it
        print "finished", self._name


if __name__ == '__main__':
    conns = []
    # all invalid addresses to see how they fail / check times
    conns.append(Connection("conn1", "www.2eg11erdhrtj.com"))
    conns.append(Connection("conn2", "www.e2ger2dh2rtj.com"))
    conns.append(Connection("conn3", "www.eg2de3rh1rtj.com"))
    conns.append(Connection("conn4", "www.ege2rh4rd1tj.com"))
    conns.append(Connection("conn5", "www.ege52drhrtj1.com"))

    for conn in conns:
        conn.start()

我不知道确切的超时时间是多少，但是运行此命令时，会发生以下情况:

I dont know exactly how long the timeout is, but when running this the following happens:

1. 所有线程开始，我得到打印输出
2. 每xx秒，一个线程显示完成，而不是一次全部显示
3. 线程按顺序完成，并非一次完成(超时=全部相同！)

所以我唯一的猜测是这与GIL有关吗?显然，线程不会并发执行任务，一次只能尝试一个连接.

So my only guess is that this has to do with the GIL? Obviously the threads do not perform their task concurrently, only one connection is attempted at a time.

有人知道解决这个问题的方法吗?

Does anyone know a way around this?

( asyncore 无济于事，我现在不希望使用 twisted )

(asyncore doesnt help, and I'd prefer not to use twisted for now) Isn't it possible to get this simple little thing done with python?

汤姆，问候

我在MacOSX上，我只是让我的朋友在linux上运行它，而他确实得到了我希望得到的结果.即使在非线程环境中，他的socket.connects()也会立即返回.即使他将套接字设置为阻塞，并且超时设置为10秒，他的所有线程也会同时完成.

I am on MacOSX, I just let my friend run this on linux, and he actually does get the results I wished to get. His socket.connects()'s return immediately, even in a non Threaded environment. And even when he sets the sockets to blocking, and timeout to 10 seconds, all his Threads finish at the same time.

有人可以解释吗?

推荐答案

在某些系统上，getaddrinfo不是线程安全的. Python认为其中一些系统是FreeBSD，OpenBSD，NetBSD，OSX和VMS.在这些系统上，Python专门为netdb(即getaddrinfo和好友)维护一个锁.

On some systems, getaddrinfo is not thread-safe. Python believes that some such systems are FreeBSD, OpenBSD, NetBSD, OSX, and VMS. On those systems, Python maintains a lock specifically for the netdb (i.e. getaddrinfo and friends).

因此，如果您不能切换操作系统，则必须使用其他(线程安全的)解析器库，例如twisted的解析器库.

So if you can't switch operating systems, you'll have to use a different (thread-safe) resolver library, such as twisted's.

这篇关于Python解释器会阻止多线程DNS请求吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！