多线程或多进程中的python socket.connect超时错误 [英] python socket.connect timeout error in mulithread or multiprocess

问题描述

类似于以下内容，我想与特定IP范围内的许多PC进行通信.

Like the following, I'd like to communicate with many PC's in a specific IP range.

My PC ---+------> Client A PC
         +------> Client B PC
         +------> Client C PC
         .................
         +------> Client Z PC

因为有太多的客户端无法通信，所以我通过多线程进行了尝试. socket.connect()持续产生超时错误. 如果我在单线程中尝试，那就没问题.

Because there are too many clients to communicate, I tried it by mulit-threading. socket.connect() continuously produces time-out error. If I try it in a single-thread, there's no problem.

我搜索了以下内容:

Python解释器会阻止多线程DNS请求吗?

说在某些平台上，套接字模块可能是线程不安全的.

saying that in some platform, socket module could be thread unsafe.

因此，我将代码更改为多处理.但是，它仍然会产生相同的错误.

So I changed my code into multi-processing. However it still produces the same error.

在下面的代码示例中，test_single()完成正常. test_mp()和test_mt()都发生超时错误.

In the following code sample, test_single() finishes normal. test_mp() and test_mt() both make time-out error.

您是否经历过这种异常行为? 测试环境为Windows XP SP3，python 2.5.4. 还尝试在python 2.6.6和2.7.0上执行相同的错误.

Have you ever experienced such abnormal behavior? The testing environment is Windows XP SP3, python 2.5.4. Also tried on python 2.6.6 and 2.7.0, same error.

import multiprocessing
import Queue
import socket
import threading

PROCESS_NUM = 5
PORT = 8888

def search_proc(ip):
    try:
        csock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        csock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        csock.settimeout(5.0)
        csock.connect((ip, PORT))
        csock.shutdown(socket.SHUT_RDWR)
        csock.close()
        return ip, "ok"
    except socket.error, msg:
        return ip, "fail", msg

def mp_connect(ip_range):
    pool = multiprocessing.Pool( PROCESS_NUM )
    for output in pool.imap_unordered(search_proc, ip_range):
        print output

def test_mp():
    ip_range = []
    for i in range(256):
        ip_range.append("192.168.123.%d"%(i,))

    mp_connect(ip_range)

def test_mt():
    def search_thread(ip_queue):
        while True:
            ip = ip_queue.get()
            print search_proc(ip)
            ip_queue.task_done()
    ip_queue = Queue.Queue()

    for i in range(256):
        ip_queue.put("192.168.123.%d"%(i,))

    for i in range(PROCESS_NUM):
        th = threading.Thread(target=search_thread, args=(ip_queue,))
        th.setDaemon(True)
        th.start()

    ip_queue.join()

def test_single():
    ip_range = []
    for i in range(256):
        print search_proc("192.168.123.%d"%(i,))

if __name__ == "__main__":
    multiprocessing.freeze_support()
    test_mp()
    #test_single()
    #test_mt()

推荐答案

David Beazley对Python GIL及其如何影响IO和多线程进行了一些出色的研究.您可以在此处，

David Beazley has done some great research around the Python GIL and how that affects IO and multithreading. You can find information about his research here, here.

这篇关于多线程或多进程中的python socket.connect超时错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！