使用pysftp将存储桶文件发送到FTP服务器 [英] Send a bucket file to FTP server using pysftp

问题描述

我正在尝试使用pysftp将文件从存储桶发送到FTP服务器.

I'm trying to send a file from a bucket to a FTP server using pysftp.

为此，我将Google云功能与python 3.7结合使用

For this I'm using Google cloud functions with python 3.7

我尝试了许多不同的方法，但总是会出错.

I have tried with many different ways, but always with an error.

1.- 以字符串形式下载文件:在本示例中，为避免文件内容出错，我将创建一个包含测试字符串"的文件，而不使用存储桶文件中的内容

1.- Downloading the file as string: in this example, to avoiding error with the file content, I will create a file which contains "test string" instead of using what is inside of the bucket file

def sftp_handler():
    myHostname = "hotsname.com"
    myUsername = "username"
    myPassword = "pass"
    try:
        keydata = b"""AAAAB..."""
        key = paramiko.RSAKey(data=decodebytes(keydata))
        cnopts = pysftp.CnOpts()
        cnopts.hostkeys.add('hotname.com', 'ssh-rsa', key)
        with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword,port=22,cnopts=cnopts) as sftp:
            print ("Connection succesfully stablished ... ")
            remoteFilePath = '/dir1/dir2/dir3/'
            sftp.putfo(StringIO('test string'), os.path.join(remoteFilePath, "OC.csv"))            
        # connection closed automatically at the end of the with-block
    except:
        print("Unexpected error in sftp_handler:")
        traceback.print_exc()

错误输出(stackdriver log)

The error output (stackdriver log )

Traceback (most recent call last): E    undefined
File "/user_code/main.py", line 36, in sftp_handler E  
sftp.putfo(StringIO('xml string'), os.path.join(remoteFilePath, "OC.csv")) E  
File "/env/local/lib/python3.7/site-packages/pysftp/__init__.py", line 474, in putfo E  
callback=callback, confirm=confirm) E  
File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 714, in putfo E  
with self.file(remotepath, "wb") as fr: E  
File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 372, in open E  
t, msg = self._request(CMD_OPEN, filename, imode, attrblock) E  
File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 813, in _request E  
return self._read_response(num) E  
File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 865, in _read_response E  
self._convert_status(msg) E 
File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 898, in _convert_status E  
raise IOError(text) 
 OSError E  

2.- 使用临时文件，其中我将从存储桶中写入文件内容

2.- Using a temp file in which I will write the content of the file from the bucket

<adding this to the previous code>
with tempfile.NamedTemporaryFile(suffix='.csv', prefix=os.path.basename(__file__)) as tf:
      tf.write(b'Hello world!')
      remoteFilePath = '/dir1/dir2/dir3/'
      sftp.put(os.path.dirname(tf.name), os.path.join(remoteFilePath, "OC.csv"))

错误日志:

 Traceback (most recent call last): E  
   File "/user_code/main.py", line 40, in sftp_handler E  
     sftp.put(os.path.dirname(tf.name), os.path.join(remoteFilePath, "OC.csv")) E  
   File "/env/local/lib/python3.7/site-packages/pysftp/__init__.py", line 364, in put E  
     confirm=confirm) E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 758, in put E  
     with open(localpath, "rb") as fl: E  
 IsADirectoryError: [Errno 21] Is a directory: '/tmp' E  

3.-正如马丁在回答中所建议的:

3.- As Martin suggested in his answer:

    with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword,port=22,cnopts=cnopts) as sftp:
        print ("Connection succesfully stablished ... ")
        remoteFilePath = '/dir1/dir2/dir3/'
        sftp.putfo(StringIO('test string'), remoteFilePath + "OC.csv")

我收到以下错误:

 Traceback (most recent call last): E  
   File "/user_code/main.py", line 36, in sftp_handler E  
     sftp.putfo(StringIO('test string'), remoteFilePath + "OC.csv") E  
   File "/env/local/lib/python3.7/site-packages/pysftp/__init__.py", line 474, in putfo E  
     callback=callback, confirm=confirm) E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 714, in putfo E  
     with self.file(remotepath, "wb") as fr: E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 372, in open E  
     t, msg = self._request(CMD_OPEN, filename, imode, attrblock) E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 813, in _request E  
     return self._read_response(num) E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 865, in _read_response E  
     self._convert_status(msg) E  
   File "/env/local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 898, in _convert_status E  
     raise IOError(text) E  
 OSError E 

4.-从存储桶下载文件，在执行云功能时创建文件，然后将此文件发送到sftp(尚未实现，因为我不确定它是否可以在云功能中创建文件)

4.- Downloading the file from the bucket, create a file while executing the cloud functions and send this file to the sftp (Not implemented yet as I'm not sure if it will work creating a file in a Cloud Function)

1. 我该怎么做?
2. 为什么失败?

---

注意:我也尝试使用FileZilla，但我也遇到错误，因此这应该与FTP服务器有关:

---

Note: I also tried using FileZilla and I also have an error, so this should be something related the FTP server:

09:00:46    Status: Connecting to sftp-staging.messagesnetwork.com...
09:01:06    Status: Connected to sftp-staging.messagesnetwork.com
09:01:06    Status: Retrieving directory listing...
09:01:06    Status: Listing directory /
09:01:07    Status: Directory listing of "/" successful
09:01:11    Status: Retrieving directory listing of "/folder1"...
09:01:37    Command:    cd "folder1"
09:01:37    Response:   New directory is: "/folder1"
09:01:37    Command:    ls
09:01:37    Error:  Connection timed out after 20 seconds of inactivity
09:01:38    Error:  Failed to retrieve directory listing
09:01:38    Status: Disconnected from server
09:01:38    Status: Connecting to sftp-staging.messagesnetwork.com...
09:02:04    Status: Connected to sftp-staging.messagesnetwork.com
09:02:09    Status: Retrieving directory listing of "/folder1"...
09:02:10    Status: Listing directory /folder1
09:02:10    Status: Directory listing of "/folder1" successful
09:02:12    Status: Retrieving directory listing of "/folder1/folder2"...
09:02:21    Status: Listing directory /folder1/folder2
09:02:23    Status: Directory listing of "/folder1/folder2" successful
09:02:24    Status: Retrieving directory listing of "/folder1/folder2/folder3"...
09:02:25    Status: Listing directory /folder1/folder2/folder3
09:02:26    Status: Directory listing of "/folder1/folder2/folder3" successful
09:02:30    Status: Connecting to sftp-staging.messagesnetwork.com...
09:02:41    Status: Connected to sftp-staging.messagesnetwork.com
09:02:45    Status: Starting upload of <...>\Desktop\folder2cf.txt
09:02:59    Command:    cd "/folder1/folder2/folder3"
09:02:59    Response:   New directory is: "/folder1/folder2/folder3"
09:02:59    Command:    put "<...>\Desktop\folder2cf.txt" "folder2cf.txt"
09:02:59    Error:  /folder1/folder2/folder3/folder2cf.txt: open for write: failure
09:02:59    Error:  File transfer failed
09:02:59    Status: Starting upload of <...>\Desktop\folder2cf.txt
09:02:59    Status: Retrieving directory listing of "/folder1/folder2/folder3"...
09:02:59    Status: Listing directory /folder1/folder2/folder3
09:03:11    Command:    put "<...>\Desktop\folder2cf.txt" "folder2cf.txt"
09:03:11    Error:  /folder1/folder2/folder3/folder2cf.txt: open for write: failure
09:03:11    Error:  File transfer failed
09:03:11    Status: Starting upload of <...>\Desktop\folder2cf.txt
09:03:11    Status: Retrieving directory listing of "/folder1/folder2/folder3"...
09:03:11    Status: Listing directory /folder1/folder2/folder3
09:03:34    Command:    put "<...>\Desktop\folder2cf.txt" "folder2cf.txt"
09:03:34    Error:  /folder1/folder2/folder3/folder2cf.txt: open for write: failure
09:03:34    Error:  File transfer failed
09:03:34    Status: Retrieving directory listing of "/folder1/folder2/folder3"...
09:03:39    Status: Listing directory /folder1/folder2/folder3
09:03:44    Status: Directory listing of "/folder1/folder2/folder3" successful

推荐答案

sftp.putfo(StringIO('test string'), os.path.join(remoteFilePath, "OC.csv"))

如果这是真正的问题，我不能说atm，但是您不应将 path.join 用于SFTP路径.SFTP始终使用正斜杠. path.join 使用本地操作系统的分隔符.可能有所不同(尤其是在Windows上).

I cannot say atm, if this is the real problem, but you should not use path.join for SFTP paths. SFTP always uses forward slash. path.join uses a separator of the local operating system. It may be different (particularly on Windows).

代码应为:

remoteFilePath = '/dir1/dir2/dir3/'
sftp.putfo(StringIO('test string'), remoteFilePath + "OC.csv")

---

sftp.put(os.path.dirname(tf.name), os.path.join(remoteFilePath, "OC.csv"))

在这里使用 path.dirname 很有意义.您要上传文件，因此必须提供要上传文件的路径，而不是其包含文件夹的路径.

Use of the path.dirname makes so sense here. You want to upload a file, so you have to give the path to a file to upload, not a path to its containing folder.

path.join 与以前一样存在相同的问题.

And there's the same problem with path.join as previously.

这篇关于使用pysftp将存储桶文件发送到FTP服务器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！