无法使用python并行化(线程化)telnet连接 [英] cannot parallelize (threads) telnet connections with python

问题描述

我有一个代码，该代码将使用telentlib连接到多个路由器，在其中运行一些代码，最后将输出写入文件.它运行顺利.

I have a code that will connect to several routers, using the telentlib, run some code within those, and finally write the output to a file. It runs smoothly.

但是，当我不得不访问许多路由器(+50)时，该任务会花费大量时间(代码是串行运行的，一次运行一台路由器).然后我考虑实现线程以加快进程.

However, when I had to access lots of routers (+50) the task consumes a lot of time (the code runs serially, one router at a time). I thought then on implementing threads in order to accelerate the process.

这几乎是代码(只是其中的一小段):

This is pretty much the code (just a snippet of it):

# We import the expect library for python
import telnetlib
import sys
import csv
import time
import threading

# --- Timers
TimeLogin = 10
TelnetWriteTimeout = 1
TelnetReadTimeout = 2
# --- CSV
FileCsv = "/home/lucas/Documents/script/file.csv"
# --- Extras
cr="\n"

# variables
IP = "A.B.C.D"
User = ["user","password"]
CliLogin = "telnet " + IP
Prompt = ["root@host.*]#"]
PromptLogin = ["login:"]
PromptLogout = ["logout"]
PromptPass = ["Password:"]
CliLine = "ls -l"

class MiThread(threading.Thread):  
    def __init__(self,num,datos):  
        threading.Thread.__init__(self)
        self.num = num
        self.datos = datos
        self.systemIP = self.datos[0]
        self.tn = telnetlib.Telnet(IP)
        self.tn.timeout = TimeLogin

        # File declaration
        self.FileOutGen = self.systemIP + "_commands"
        self.FileOutSend = self.systemIP + "_output"
        self.FileOutRx = self.systemIP + "_rx"      
        self.fg = open(self.FileOutGen,"w")
        self.fr = open(self.FileOutRx,"a")
        self.fs = open(self.FileOutSend,"w")

    def run(self):  
        print "Soy el hilo", self.num
        self.telnetLogin()
        self.runLs()
        self.telnetLogout()

    def telnetLogin(self):      
        i=self.tn.expect(PromptLogin)
        print i
        if i:
            writeTelnet(User[0],TelnetWriteTimeout)
            j=self.tn.expect(PromptPass)
            print j
            if j:
                writeTelnet(User[1],TelnetWriteTimeout)

    def telnetLogout(self):
        i=self.tn.expect(Prompt)
        if i:
            writeTelnet("exit",TelnetWriteTimeout)
            j=self.tn.expect(PromptLogout)
            if j:
                print "Logged out OK from SAM"

    def runLs(self):
        writeLog("Prueba de Ls " + self.systemIP)
        self.writeCsv(self.systemIP,1)
        i=self.tn.expect(Prompt,TimeLogin)
        print i
        if i:
            # Prompt
            CliLine = "ls -l "
            writeTelnet(CliLine,TelnetWriteTimeout)

    def writeCsv(self,inText,lastIn):
        lock.acquire(1)
        if lastIn==0:
            fc.write(inText + ",")
        elif lastIn==1:
            fc.write(inText + "\n")
        lock.release()

def writeTelnet(inText, timer):
    tn.write(inText + cr)
    time.sleep(timer)

def writeLog(inText):
    print (inText)
    t.fs.write("\n" + inText)

def printConsole(inText):
    print (inText)

oFile = csv.reader(open(FileCsv,"r"), delimiter=",", quotechar="|")

lock = threading.Lock()
routers = list(oFile)
threads_list = []

for i in range(len(routers)):
    FileOutCsv = "00_log.csv"

    # creating output file
    fc = open(FileOutCsv,"a")

    # running routine
    t = MiThread(i,routers[i])
    threads_list.append(t)
    t.start()

...一切运行良好，但是没有时间收益，因为t.join()将强制线程在运行下一个线程之前完成！

... everything runs nicely, but there is no gain in time since the t.join() will force a thread to complete before running the next one!

对线程进行序列化的事实(意味着使用t.join())使我认为某些内存空间正在共享，因为当我确实要对它们进行并行化(注释掉t.join()时，就出现了问题) ).

The fact of serializing the threads (meaning, using the t.join()) makes me think that some memory spaces are being shared because the problem arises when I do want to parallelize them (commenting out the t.join()).

我在做错什么吗?如果需要，我可以提供更多信息，但我真的不知道到目前为止我做错了什么...

Is there anything wrong that I'm doing? I can provide more information if needed but I really do not know what is it that I'm doing wrong so far ...

推荐答案

因此，在进行一些挖掘之后，发现了错误.

So after some digging, the error was found.

之前，函数writeTelnet()是在类外部声明的.一旦移入其中并被其余部分(即self.writeTelnet())正确引用，一切就可以按预期进行.

Before, the function writeTelnet() was declared outside the class. Once moved inside of it and properly being referenced by the rest (ie: self.writeTelnet()) everything worked so far as expected.

这是新代码的摘录:

class MiThread(threading.Thread):  
    def __init__(self,num,datos):  
        threading.Thread.__init__(self)
        self.num = num
        self.datos = datos
        self.systemIP = self.datos[0]
        self.tn = telnetlib.Telnet(IP)
        self.tn.timeout = TimeLogin

        # File declaration
        self.FileOutGen = self.systemIP + "_commands"
        self.FileOutSend = self.systemIP + "_output"
        self.FileOutRx = self.systemIP + "_rx"      
        self.fg = open(self.FileOutGen,"w")
        self.fr = open(self.FileOutRx,"a")
        self.fs = open(self.FileOutSend,"w")

    [ . . . ]

    def writeTelnet(self,inText, timer):
        self.tn.write(inText + ch_cr)
        time.sleep(timer)

oFile = csv.reader(open(FileCsv,"r"), delimiter=",", quotechar="|")
routers = list(oFile)

for i in range(len(routers)):   
    # creating output file
    fc = open(FileOutCsv,"a")

    # running routine
    t = MiThread(i,routers[i])
    t.start()

现在(至少对我来说)很清楚:由于不同的线程在某个时候需要写入其telnet的连接，因此他们需要明确地进行标识.我发现这样做的唯一方法是将函数包含在类中.

And it's now clear (at least to me): Since the different threads need at some point to write into their telnet's connection, they need unambiguously to identify it. The only way that I've found to do it is to include the function inside the class.

谢谢大家

卢卡斯

这篇关于无法使用python并行化(线程化)telnet连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！