如何将从文件读取的工作分配到线程? [英] How can I divide work read from a file to threads?

问题描述

我有一个主机列表和一个端口列表.我想运行 X 线程，每次都会抓取一个主机并尝试连接到这些端口.我得到了连接部分，我的问题是如何穿线.当我进行一些研究时，我对线程、多进程和 asyncio 感到非常困惑.什么是最佳和/或最简单的库?

I have a list of hosts and a list of ports. I want to run X threads that will grab a single host every time and try to connect to these ports. I got the connection part covered, my problem is how to thread it. When I did some research I got really confused between threading, multiproccessing and asyncio. What would be the most optimal and/or easy library?

现在我的伪代码没有线程代码是:

right now my pseudo code no threaded code is:

def getHosts():
    hosts = open("hostList.txt").read().splitlines()
    return hosts
def getPorts():
    ports = open("portList.txt").read().splitlines()

hosts = getHosts()
ports = getPorts()
for host in hosts
    for port in ports
        ---the connection code---

我认为我的一般想法是取列表的长度，将其除以线程数并创建一个线程，该线程将从 thread_number*result_of_divide 运行直到 (thread_number+1)*result_of_divide.

I think my general idea is to take the length of the list, divide it by number of threads and create a thread that will run from thread_number*result_of_divide until (thread_number+1)*result_of_divide.

推荐答案

ThreadPoolExecutor.

import concurrent.futures

MAX_THREAD_COUNT = 10


def read_text_file(filename):
    with open(filename, "r") as f:
        return f.read().splitlines()


def check_port(host, port):
    pass


hosts = read_text_file("hostList.txt")
ports = read_text_file("portList.txt")


with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_THREAD_COUNT) as executor:
    results = {}
    for host in hosts:
        for port in ports:
            results[executor.submit(check_port, host, port)] = "{}:{}".format(host, port)

for result in concurrent.futures.as_completed(results):
    host_and_port = results[result]
    try:
        returned_data = result.result()
    except Exception as e:
        print("\"{}\" exception have been caused while checking {}".format(e, host_and_port))
    else:
        print("Checking of {} returned {}".format(host_and_port, returned_data))

P.S. 代码可能不是 100% 正确，我还没有实际"检查过.

P.S. Code could be not 100% correct, I've not checked it "in action".

这篇关于如何将从文件读取的工作分配到线程?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！