Azure功能使用FluentFTP将文件从FTP复制到Blob存储 [英] Azure function to copy files from FTP to blob storage using FluentFTP

问题描述

我有一个FTP源，每天都会添加文件，我需要使用天蓝色功能中的FluentFTP库每天将文件从FTP复制到blob存储

I have an FTP source that adds daily file and I need to copy the files on daily basis from FTP to blob storage using FluentFTP library in azure function

我在Azure函数中使用C#，我做了所有编码部分，但是缺少从FTP下载文件以将其直接复制到blob目标的情况.

I am using C# in Azure function, and I did all the coding part but missing to download the file from FTP to copy it directly to the blob destination.

//#r "FluentFTP"
#r "Newtonsoft.Json"
#r "System.Data"
#r "Microsoft.WindowsAzure.Storage"

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Collections.Generic;
using System.Globalization;
using System;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;

using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
using FluentFTP;

public static async Task<IActionResult> Run(HttpRequest req, ILogger log)
{
    string blobConnectionString = "ConnectionString";

    // Gestione BLOB Storage
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(blobConnectionString);
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    CloudBlobContainer container = blobClient.GetContainerReference("out");

    using (FtpClient conn = new FtpClient()) 
    {
        conn.Host = "ftp server";
        conn.Credentials = new NetworkCredential("username", "pass");

        // get a list of files and directories in the "/OUT" folder
        foreach (FtpListItem item in conn.GetListing("/OUT")) 
        {
            // if this is a file and ends with CSV
            if (
            item.Type == FtpFileSystemObjectType.File
            &&
            item.FullName.ToLower().EndsWith(".csv")
            )
            {

                string yyyy = item.FullName.Substring(10,4);
                string mm   = item.FullName.Substring(14,2);
                string dd   = item.FullName.Substring(16,2);
                var fileName = "out/brt/" + yyyy + "/"+ mm + "/"+ dd + "/" + item.Name;

                CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
                // download the file
                conn.DownloadFile( blockBlob , item.FullName);
            }
        }

        return new OkObjectResult($"Hello");
    }
}

如果我可以将blob容器用作FluentFTP函数的目标，那将是最好的选择，但是这样我会收到一个错误，即我正在使用的blob块不是目标

If I can use a blob container as a destination for the FluentFTP function then it would be the best, but this way I am getting the error that this blob block that I am using is not an a destination

这是我遇到的错误

无法从"Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob"转换为字符串"

cannot convert from 'Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob' to 'string'

我不知道是否还有另一种本地下载文件的方法，因此我可以将其上传到Blob，而不是使用此client.DownloadFile函数.

I don't know if there is another way to download the file locally so I can upload it to the blob instead of using this client.DownloadFile function.

推荐答案

使用Fluent FTP可能无法实现.

It's probably not possible with Fluent FTP.

有FtpClient.Download方法采用Stream.

public bool Download(Stream outStream, string remotePath, IProgress<double> progress = null)

但是看起来好像没有API可以获取大量上传流" .

But it does not look like there's API to get "blob upload Stream".

相反，您无法从FluentFTP获得""FTP下载流" (可以与blob API一起使用).

Conversely, you cannot get "FTP download Stream" from FluentFTP (which you would be able to use with blob API).

但是您可以使用本机.NET FtpWebRequest FTP客户端，该客户端具有API来获取"FTP下载流" :

But you can use native .NET FtpWebRequest FTP client, which has API to get "FTP download Stream":

public static async System.Threading.Tasks.Task RunAsync([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
    var ftpUserName = "xxxxxx";
    var ftpPassword = "xxxxxxxxx";
    var filename = "test.png";
    string blobConnectionString = "xxxxxxx";
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(blobConnectionString);
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    CloudBlobContainer container = blobClient.GetContainerReference("myblobcontainer");
    FtpWebRequest fileRequest = (FtpWebRequest)WebRequest.Create("ftp://xxxxxx/" + filename);
    fileRequest.Method = WebRequestMethods.Ftp.DownloadFile;
    fileRequest.Credentials = new NetworkCredential(ftpUserName, ftpPassword);
    FtpWebResponse fileResponse = (FtpWebResponse)fileRequest.GetResponse();
    Stream fileStream = fileResponse.GetResponseStream();
    CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);

    await blockBlob.UploadFromStreamAsync(fileStream);
    log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}

这篇关于Azure功能使用FluentFTP将文件从FTP复制到Blob存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！