AWS Lambda中的FTP - 下载文件的问题（异步/等待） [英] FTP in AWS Lambda - Issues Downloading Files (Async/Await)

问题描述

我一直在努力使用各种FTP节点模块来尝试在AWS Lambda中运行任何工作。最好的和最流行的似乎是Basic-FTP，它也支持async / await。但是当我在FTP函数下添加任何代码时，我无法下载文件。

我不想在FTP异步函数中添加fs函数当我添加下面的代码时，我需要解决导致中断的原因，并且还有其他位代码可以添加并使用下载的文件，以及稍后的内容：

FTP成功 - 当async函数仅在其下方没有fs代码的情况下使用时

FTP FAILURE - 添加fs readdir / readFile函数或下面的任何其他代码

错误 错误：ENOENT：没有这样的文件或目录，打开'/tmp/document.txt'

以下是完整的CloudWatch日志：

我的文件里面只包含一个你好。您可以在日志中看到它。

请记住，在Lambda函数中，将任何内容下载到/ tmp时，您有512MB的限制。您可以在 docs 中查看限制

I have been struggling with various FTP Node modules to try and get anything working in AWS Lambda. The best and most popular seems to be "Basic-FTP" that also supports async/await. But I just cannot get it to download files when any code is added beneath the FTP function.

I don't want to add the fs functions within the FTP async function as I need to solve what is causing the break when any code below is added and I also have other bits of code to add and work with the downloaded file and it's content later:

FTP SUCCESS - When the async function is used only with no fs code beneath it

FTP FAILURE - Adding the fs readdir/readFile functions or any other code below

ERROR Error: ENOENT: no such file or directory, open '/tmp/document.txt'

https://github.com/patrickjuchli/basic-ftp

const ftp = require("basic-ftp");
const fs = require("fs");
var FileNameWithExtension = "document.txt";
var ftpTXT;

exports.handler = async (event, context, callback) => {

    example();

    async function example() {
        const client = new ftp.Client();
        //client.ftp.verbose = true;
        try {
            await client.access({
                host: host,
                user: user,
                password: password,
                //secure: true
            });
            console.log(await client.list());
            await client.download(fs.createWriteStream('/tmp/' + FileNameWithExtension), FileNameWithExtension);
        }
        catch (err) {
            console.log(err);
        }
        client.close();
    }

    // Read the content from the /tmp/ directory to check FTP was succesful
    fs.readdir("/tmp/", function (err, data) {
        if (err) {
            return console.error("There was an error listing the /tmp/ contents.");
        }
        console.log('Contents of AWS Lambda /tmp/ directory: ', data);
    });

    // Read TXT file and convert into string format
    fs.readFile('/tmp/' + FileNameWithExtension, 'utf8', function (err, data) {
        if (err) throw err;
        ftpTXT = data;
        console.log(ftpTXT);
    });

    // Do other Node.js coding with the downloaded txt file and it's contents
};

解决方案

The problem is that you are getting lost when creating an async function inside your handler. Since example() is async, it returns a Promise. But you don't await on it, so the way it has been coded, it's kind of a fire and forget thing. Also, your Lambda is being terminated before your callbacks are triggered, so even if it got to download you would not be able to see it.

I suggest you wrap your callbacks in Promises so you can easily await on them from your handler function.

I have managed to make it work: I have used https://dlptest.com/ftp-test/ for testing, so change it accordingly. Furthermore, see that I have uploaded the file myself. So if you want to replicate this example, just create a readme.txt on the root of your project and upload it. If you already have this readme.txt file on your FTP server, just delete the line where it uploads the file.

Here's a working example:

const ftp = require("basic-ftp");
const fs = require("fs");
const FileNameWithExtension = "readme.txt";

module.exports.hello = async (event) => {

  const client = new ftp.Client();
  try {
    await client.access({
      host: 'ftp.dlptest.com',
      user: '[e-mail]',
      password: 'puTeT3Yei1IJ4UYT7q0r'
    });
    console.log(await client.list());
    await client.upload(fs.createReadStream(FileNameWithExtension), FileNameWithExtension)
    await client.download(fs.createWriteStream('/tmp/' + FileNameWithExtension), FileNameWithExtension);
  }
  catch (err) {
    console.log('logging err')
    console.log(err);
  }
  client.close();

  console.log(await readdir('/tmp/'))

  console.log(await readfile('/tmp/', FileNameWithExtension))

  return {
    statusCode: 200,
    body: JSON.stringify({message: 'File downloaded successfully'})
  }

};

const readdir = dir => {
  return new Promise((res, rej) => {
    fs.readdir(dir, function (err, data) {
      if (err) {
        return rej(err);
      }
      return res(data)
    });
  })
}

const readfile = (dir, filename) => {
  return new Promise((res, rej) => {
    fs.readFile(dir + filename, 'utf8', function (err, data) {
      if (err) {
        return rej(err);
      }
      return res(data)
    })
  })
}

Here is the output of the Lambda function:

And here are the complete CloudWatch logs:

My file contains nothing but a 'hello' inside it. You can see it on the logs.

Do keep in mind that, in Lambda Functions, you have a 512MB limit when downloading anything to /tmp. You can see the limits in the docs

这篇关于AWS Lambda中的FTP - 下载文件的问题（异步/等待）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！