在python中使用paramiko打开远程文件缓慢 [英] Open a remote file using paramiko in python slow
问题描述
我正在使用paramiko在python中打开一个远程sftp文件.使用paramiko返回的文件对象,我正在逐行读取文件并处理信息.与使用os中的python内置方法"open"相比,这似乎确实很慢.以下是我用来获取文件对象的代码.
I am using paramiko to open a remote sftp file in python. With the file object returned by paramiko, I am reading the file line by line and processing the information. This seems really slow compared to using the python in-built method 'open' from the os. Following is the code I am using to get the file object.
使用paramiko(慢2倍)-
Using paramiko (slower by 2 times) -
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(myHost,myPort,myUser,myPassword)
sftp = client.open_sftp()
fileObject = sftp.file(fullFilePath,'rb')
使用操作系统-
import os
fileObject = open(fullFilePath,'rb')
我错过了什么吗?有没有一种方法可以使paramiko fileobject的读取方法与使用os fileobject的方法一样快?
Am I missing anything? Is there a way to make the paramiko fileobject read method as fast as the one using the os fileobject?
谢谢!
推荐答案
您的问题很可能是由文件作为远程对象引起的.您已经在服务器上打开了它,一次只请求一行-因为它不是本地的,所以每个请求所花的时间比文件放在硬盘上要长得多.最好的替代方法可能是先使用Paramiko的 SFTP get
.
Your problem is likely to be caused by the file being a remote object. You've opened it on the server and are requesting one line at a time - because it's not local, each request takes much longer than if the file was sitting on your hard drive. The best alternative is probably to copy the file down to a local location first, using Paramiko's SFTP get
.
完成此操作后,您可以使用os.open
从本地位置打开文件.
Once you've done that, you can open the file from the local location using os.open
.
这篇关于在python中使用paramiko打开远程文件缓慢的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!