FTP:复制、检查完整性和删除 [英] FTP: copy, check integrity and delete

问题描述

我正在寻找一种使用 ftp 或 lftp 连接到远程服务器的方法，并确保执行以下步骤:

I am looking for a way to connect to a remote server with ftp or lftp and make sure the following steps:

1. 将文件从 FTP 服务器复制到我的本地计算机.

1. Copy files from FTP server to my local machine.

检查下载的文件是否正常(即md5checksum).

Check if the downloaded files are fine (i.e. md5checksum).

如果下载正常，则从 FTP 服务器中删除下载的文件.

If the download was fine then delete the downloaded files from the FTP server.

这个例程每天都会从我的本地机器上执行.这样做的最佳选择是什么?是否有一种工具可以抽象出所有 3 个步骤?

This routine will be executed each day from my local machine. What would be the best option to do this? Is there a tool that makes abstraction of all the 3 steps ?

我在客户端和服务器机器上都运行 Linux.

I am running Linux on both client and server machines.

更新:此外，我还有一个文本文件，其中包含 FTP 服务器上的文件与其 MD5sum 之间的关联.它们是在 FTP 服务器端计算的.

Update: Additionally, I have also a text file that contains the association between the files on the FTPserver and their MD5sum. They were computed at the FTP server side.

推荐答案

首先，确保你的远程服务器完全支持校验和计算.许多人没有.我相信甚至没有标准的 FTP 命令来计算远程文件的校验和.有很多提案，也有很多专有解决方案.

First, make sure your remote server supports the checksum calculation at all. Many do not. I believe there's even no standard FTP command to calculate a checksum of a remote file. There were many proposals and there are many proprietary solutions.

最新提案是:
https://datatracker.ietf.org/doc/html/草稿-bryan-ftpext-hash-02

所以即使你的服务器支持校验和计算，你也必须找到支持相同命令的客户端.

So even if your server supports checksum calculation, you have to find a client that supports the same command.

一些可用于计算校验和的命令有:XSHA1、XSHA256、XSHA512、XMD5、MD5、XCRC 和 HASH.

Some of the commands that can be used to calculate checksum are: XSHA1, XSHA256, XSHA512, XMD5, MD5, XCRC and HASH.

您可以使用 WinSCP 进行测试.WinSCP 支持所有前面提到的命令.测试其校验和计算函数或checksum 脚本命令.如果它们有效，启用日志记录并检查 WinSCP 对您的服务器使用的命令和语法.

You can test that with WinSCP. The WinSCP supports all the previously mentioned commands. Test its checksum calculation function or the checksum scripting command. If they work, enable logging and check, what command and what syntax WinSCP uses against your server.

ftp(无论是Windows还是*nix版本)和lftp都不支持校验和计算，只允许自动验证下载的文件.

Neither the ftp (neither Windows nor *nix version) nor the lftp support checksum calculation, let only automatic verification of downloaded file.

我什至不知道有任何其他客户端可以自动验证下载的文件.

I'm not even aware of any other client that can automatically verify downloaded file.

您绝对可以借助一些功能丰富的客户端编写脚本.

You can definitely script it with a help of some feature-rich client.

在 OP 指定他/她在 Linux 上之前，我已经写了这个答案.我保留 Windows 解决方案以防它帮助其他人.

在 Windows 上，您可以使用 PowerShell 使用 WinSCP .NET 程序集编写脚本..p>

On Windows, you could script it with PowerShell using WinSCP .NET assembly.

param (
    $sessionUrl = "ftp://username:[e-mail]/",
    [Parameter(Mandatory)]
    $localPath,
    [Parameter(Mandatory)]
    $remotePath,
    [Switch]
    $pause = $False
)
 
try
{
    # Load WinSCP .NET assembly
    Add-Type -Path (Join-Path $PSScriptRoot "WinSCPnet.dll")
 
    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.ParseUrl($sessionUrl);
 
    $session = New-Object WinSCP.Session
 
    try
    {
        # Connect
        $session.Open($sessionOptions)
        
        Write-Host "Downloading $remotePath to $localPath..."
        $session.GetFiles($remotePath, $localPath).Check();

        # Calculate remote file checksum
        $buf = $session.CalculateFileChecksum("sha-1", $remotePath)
        $remoteChecksum = [BitConverter]::ToString($buf)
        Write-Host "Remote file checksum:     $remoteChecksum"

        # Calculate local file checksum
        $sha1 = [System.Security.Cryptography.SHA1]::Create()
        $localStream = [System.IO.File]::OpenRead($localPath)
        $localChecksum = [BitConverter]::ToString($sha1.ComputeHash($localStream))
        Write-Host "Downloaded file checksum: $localChecksum"

        # Compare cheksums
        if ($localChecksum -eq $remoteChecksum)
        {
            Write-Host "Match, deleting remote file"

            $session.RemoveFiles($remotePath).Check();
            $result = 0
        }
        else
        {
            Write-Host "Does NOT match"
            $result = 1
        }
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
    }
}
catch [Exception]
{
    Write-Host "Error: $($_.Exception.Message)"
    $result = 1
}
 
# Pause if -pause switch was used
if ($pause)
{
    Write-Host "Press any key to exit..."
    [System.Console]::ReadKey() | Out-Null
}
 
exit $result

你可以像这样运行它:

powershell -file checksum.ps1 -remotePath ./file.dat -localPath C:pathfile.dat

这部分基于 通过 SFTP/与本地文件验证远程文件的校验和的 WinSCP 示例FTP协议.

(我是 WinSCP 的作者)

这个问题后来被编辑说 OP 有一个带有校验和的文本文件.这使它成为一个完全不同的问题.只需下载文件，计算本地校验和并将其与文本文件中的校验和进行比较.如果匹配，则删除远程文件.

The question was later edited to say that OP has a text file with a checksum. That makes it a completely different question. Just download the file, calculate local checksum and compare it to the checksum you have in the text file. If they match, delete the remote file.

这篇关于FTP:复制、检查完整性和删除的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！