Python3线程，尝试同时ping通多个IP/测试端口 [英] Python3 threading, trying to ping multiple IPs/test port simultaineously

问题描述

下面的完整(无效)代码

Full (non-working) code below

完整(工作时，不带线程)代码在这里: http://pastebin.com/KUYzNtT2

Full (working, w/o threading) code here: http://pastebin.com/KUYzNtT2

我编写了一个小脚本，该脚本执行以下操作:

I've written a small script that does the following:

1. 从数据库中提取网络信息
2. 在cidr中对每个IP进行ping(即-192.168.0.0/24)；如果启动，请测试以查看某个端口是否已打开
3. 显示结果

这很好，但是我想实现线程化以使脚本运行得更快；因为我要扫描的IP数以千计，所以要花很多时间.

This is working fine, but I'd like to implement threading to make the script run faster; as is I have thousands of IPs to scan and it takes forever.

我玩过线程教程，但似乎无法掌握如何在脚本中实现它.

I've played around with threading tutorials but can't seem to grasp how to implement it in my script.

任何想法或建议都值得赞赏.

Any thoughts or suggestions are appreciated.

我根据此指南转向了不同的方向: http ://chriskiehl.com/article/parallelism-in-one-line/

I went in a different direction based on this guide: http://chriskiehl.com/article/parallelism-in-one-line/

现在我运行程序并得到:File "port_test.py", line 39, in display_results for (client, location, cidr) in results: ValueError: too many values to unpack (expected 3) 我不明白为什么.有想法吗?

Now I run the program and get: File "port_test.py", line 39, in display_results for (client, location, cidr) in results: ValueError: too many values to unpack (expected 3) and I don't understand why. Thoughts?

**我想我弄清楚了它为什么失败，看起来像pool.map期望只有一个数据点.如果仅查询DB而不是CIDR的其他两列，则程序将开始吐出数据(比以前快得多).因此，现在我需要弄清楚如何将其他两列添加到结果中，然后对结果进行排序以使它们有意义(我认为这是合理的，结果没有顺序)

** I think I figured out why it failed, looks like pool.map expects only one data point. If I only query the DB for CIDRs instead of the other two columns, the program starts spitting out data (MUCH faster than before). So now I need to figure out how to add the other two columns to the results, then sort the results to they make sense (there's no order to the results, which I suppose makes sense)

#! /usr/bin/python
# Import modules
import socket
import subprocess
import ipaddress
import mysql.connector
import configparser
import logging
import coloredlogs
from multiprocessing.dummy import Pool as ThreadPool

#logging.basicConfig(format='%(levelname)s:%(message)s',level=logging.INFO)


coloredlogs.install(level='DEBUG')
coloredlogs.DEFAULT_LOG_FORMAT = '%(asctime)s -- %(message)s'
# read from the config file
config = configparser.ConfigParser()
config.read('config.ini')
db=config['mysql']
net=config['network']
port = int(net['port'])

# create the connection, connect, and setup the query
cnx = mysql.connector.connect(user=db['user'], database=db['database'], password=db['password'])
cursor = cnx.cursor()

query = ("select fw.net_cidr as cidr "
        "from firewalls fw "
            "left join clients c on c.id = fw.client_id "
            "left join locations l on l.id = fw.location_id "
                "where fw.net_cidr <> '' and c.active = '1' and fw.active = '1'")

cursor.execute(query)
results = cursor.fetchall()

def display_results(results):
# execute and display the results
    for (cidr) in results:
            logging.info("{} --> ".format(cidr))
            try:
                # Prompt the user to input a network address
                net_addr = str(cidr)

                # Create the network
                ip_net = ipaddress.ip_network(net_addr)

                 # Get all hosts on that network
                all_hosts = list(ip_net.hosts())
            except ValueError as e:
                logging.warning(e)
                continue

            # For each IP address in the subnet, test to see if port 3389 is open
            for i in range(len(all_hosts)):
                sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                sock.settimeout(.25)
                result = sock.connect_ex((str(all_hosts[i]),port))
                if result == 0:
                        logging.info(str(all_hosts[i]) + ": " + net['port'] + " is open")
            else:
                    logging.debug(str(all_hosts[i]) + ": " + net['port'] + " is not open")

# make a pool of workers
pool = ThreadPool(4)

# ping the cidrs in their own thread
pool.map(display_results, results)
pool.close()
pool.join()

# close the database connection
cursor.close()
cnx.close()

推荐答案

最初获取所有数据并将其存储在Queue中.

Grab all the data initially and store it in a Queue.

创建一个连续运行的功能，直到Queue为空(即while my_queue.empty() is False.

Create a function that runs continuously until the Queue is empty (i.e. while my_queue.empty() is False.

使用Queue的get()方法抓取Queue中的第一个对象.

Grab the first object in the Queue with Queue's get() method.

然后处理它.

根据需要初始化任意数量的线程，它们将一直执行到Queue为空为止.

Initialize as many threads as you want, they will execute until the Queue is empty.

这篇关于Python3线程，尝试同时ping通多个IP/测试端口的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！