paramiko的多线程 [英] multithreading for paramiko

问题描述

上下文

如果有一个脚本连接到服务器，然后卷曲本地主机以获取我需要的信息

If have a script that connects to servers and then curls the localhost for the information I need

问题

我的问题是，我需要从大约200台服务器中获取信息(我使用的方法)，大约需要15分钟才能完成，这虽然不错，但是我想做的事情更加高级，如果我可以钉多线程，我可以实现更多.

My issue is that I have around 200 servers I need to grab information from, the method I use, takes about 15 minutes to finish, which isn't bad but the stuff I'm wanting to do is more advanced and if I can nail multithreading I can achieve more.

所需结果

我只想拥有5-10个工作人员的线程池，以便我可以更快地获取所需的信息.

I just want to have a threadpool of 5-10 workers so I can grab the information I need quicker.

代码

from Queue import Queue
from multiprocessing.pool import ThreadPool
import threading, os, sys
import socket
from threading import Thread
#socket.setdefaulttimeout(5)
from time import sleep
import paramiko
hosts = []
hostnames = []
def capture_travelinfo(host):
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    print('Connecting to ' + host)
    ssh.connect(host, username='root', key_filename='...')
    print('Connected to ' + host)
    stdin, stdout, stderr = ssh.exec_command('curl localhost:4000')
    stdin.close()
    ssh.close()

def main():
    try:
        file = open('IPs.txt')
        threads = []
        content = file
        for x in content:
            fields = x.strip().split()
            UNIT = [fields[0], fields[1]]
            hosts.append(UNIT)
        for x in hosts:
            host = x[1]
            hostnames.append(x[1])
            ip = x[0]
            pool = ThreadPool(5)
            results = pool.map(capture_travelinfo, hostnames)
            pool.close()
            pool.join()
            print(results)

以前的尝试

我浏览了SO，发现了一些相关的东西，但是所有有用的材料都没有线程池，结果我最终一次连接到大约200个主机，这不是很好.

I've had a look around SO and found some stuff that relates but all the useful material doesn't include thread pools and as a result I end up connecting to about 200 hosts at once which isn't good.

推荐答案

from multiprocessing.pool import ThreadPool
import paramiko
hosts = []
hostnames = []


def capture_travelinfo(host):
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    print('Connecting to ' + host)
    ssh.connect(host, username='root', key_filename='...')
    print('Connected to ' + host)
    stdin, stdout, stderr = ssh.exec_command('curl localhost:4000')
    stdin.close()
    ssh.close()


def main():
    ips = open('IPs.txt')
    pool = ThreadPool(5)
    for ip in ips:
        fields = ip.strip().split()
        UNIT = [fields[0], fields[1]]
        hosts.append(UNIT)
    for ip in hosts:
        hostnames.append(ip[1])
    results = pool.map(capture_travelinfo, hostnames)
    pool.close()
    pool.join()
    print(results)

由于您的代码确实很糟糕，因此我无法耐心仔细调试它.这个版本几乎可以使用，以后自己检查一下即可.

As your code is really bad, I am running out of patience to carefully debug it. This version will almost work, just have a check later by yourself.

这篇关于paramiko的多线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！