AWS Lambda中的流转换导致结束后写入错误 [英] Stream transformations in AWS Lambda result in write after end error

问题描述

我正在尝试使用Fastcsv nodejs包转换CSV文件.有趣的是，这些代码片段在我的系统上可以很好地本地运行.但是，如果我尝试将其与AWS Lambda程序集成，则会给出结束后写入错误.

I am trying to transform a CSV file using Fastcsv nodejs package. Interestingly, the code snippet works locally on my system well. However, if I try to integrate it with AWS lambda program it gives write after end error.

var stream = fs.createReadStream(s3EventInfo.inputDownloadLoc)
.pipe(csv.parse({headers: true}))
//pipe the parsed input into a csv formatter
.pipe(csv.format({headers: true}))
//Using the transfrom function from the formatting stream
.transform(function(row, next){
    transformLine(row, next);
})
.pipe(fs.createWriteStream(s3EventInfo.outputFileLoc))
.on("end", function(){
    callback();
});

这是AWS日志中的错误.

Here is the error in aws logs..

Error: write after end
at writeAfterEnd (_stream_writable.js:133:12)
at PassThrough.Writable.write (_stream_writable.js:181:5)
at write (_stream_readable.js:602:24)
at flow (_stream_readable.js:611:7)
at _stream_readable.js:579:7
at process._tickDomainCallback (node.js:486:13)

请帮助您理解和解决该问题.

Please help in understanding and resolving the issue.

推荐答案

Lambda函数已部署并按需运行，但是如果尚未销毁对同一个lambda函数的其他调用，则可以在现有实例上运行.您无法控制它，但是需要确保您的代码可以正确处理它.

Lambda functions are deployed and run on demand, but additional calls to the same lambda function MAY run on the existing instance if it has not been destroyed. You can't control this but you need to ensure that you code can correctly handle it.

如果您的流是在全局范围内定义的，那么您的问题是那些后续调用正在重用已经接收到结束"事件的流.

If your stream is being defined in the global scope then your problem is that those subsequent calls are reusing the stream that has already received an 'end' event.

您需要封装流，以便为每个调用实例化新的流.

You need to encapsulate the streams so that they are instantiated new for each call.

这篇关于AWS Lambda中的流转换导致结束后写入错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！