如何在不受AWS的Linux AMI限制的情况下，将SoX二进制文件与对NodeJS AWS Lambda函数的MP3支持一起打包? [英] How to package SoX binary with MP3 support for a NodeJS AWS Lambda Function with limitations of AWS's Linux AMI?

问题描述

过去3个月来我一直在研究此问题，并且完全陷入僵局.

I've been working on this issue for the past 3 months and am completely stuck.

我正在尝试打包NodeJS AWS Lambda函数，该函数将使用 SoX 及其转换音频的依赖项文件到MP3.通过遵循 和此处.我最终将此代码添加到Lambda函数调用的开头，以更新process.env PATH变量，以包括自定义二进制文件的路径.

I am trying to package up my NodeJS AWS Lambda function that will use SoX and it's dependencies to convert audio files to MP3. I can get my code to recognize the custom location of the SoX binary by following instructions referenced here, and here. I ended up adding this code to the start of my Lambda function call to update the process.env PATH variable to include the path to the custom binary.

process.env['PATH'] = process.env['PATH'] + ':' 
+ path.join(process.env['LAMBDA_TASK_ROOT'], 'binaries');

这导致我的process.env PATH更新为如下所示:

This results in my process.env PATH updating to look like this:

/usr/local/lib64/node-v4.3.x/bin:/usr/local/bin:/usr/bin/:/bin:/var/task/binaries

看起来正确，binaries是包含我编译的sox二进制文件的目录.

Which looks correct, binaries is the directory that contains the sox binary I compiled.

由于我使用的是NodeJS，因此必须修改 sox-audio NPM模块因此它为child_process使用了更新的process.env变量 exec 和生成调用.这样可以使代码找到二进制文件，但是在执行过程中仍然出现错误.

Since I'm using NodeJS, I had to modify the sox-audio NPM module so it used the updated process.env variable for the child_process exec and spawn calls. This allows the code to find the binary, but I'm still getting an error during execution.

Sox process exited with code 127 and signal null

我知道，尽管它可以找到我包含的SoX二进制文件，但它找不到与SoX一起使用的命令，但是如果没有更多信息，我将无法确定它是什么.我当时以为是因为我不确定是否包含所有文件才能使二进制文件正常工作.

I understand that although it can find the SoX binary I included, it is not able to find a command used with SoX, but without more information I can't tell what it is. I was thinking it was because I am not sure if I have all the files included for the binary to work.

为了制作具有MP3支持的SoX的干净编译版本，我创建了一个新的EC2 linux实例，然后按照提供的说明进行操作

In an attempt to make a clean compiled build of SoX with MP3 support, I created a fresh EC2 linux instance and then followed the instructions provided here.

我逐行检查以确保可以正常工作，并安装了一些依赖项以启用编译后(例如

I went line by line to ensure I could get it working, and after installing a few dependencies to enable compiling (such as developer tools), and by exporting the build PATH with export PATH=$PATH:/usr/local/bin I was able to get a full build installed with MP3 support. I tested it and it works exactly like how I need it.

由于AWS Lambda Functions与AWS EC2实例在同一精简版Linux(Amazon Linux AMI)上运行，因此，理论上，如果我可以导出SoX构建并将其包含在Lambda软件包中，那么我应该能够它可以正常工作.

Since AWS Lambda Functions run on the same stripped down version of Linux (Amazon Linux AMI) as AWS EC2 instances, theoretically, if I could export the SoX build and include it in my Lambda Package, then I should be able to get it to work.

那是我遇到麻烦的地方.由什么构成建筑本身? /usr/local/bin中有一个SoX linux可执行文件，它是一个文件，但是那里还有很多其他文件，这些文件似乎都与使SoX及其依赖项起作用有关.这是我拥有的有效版本中/usr/local/bin中文件的列表.

That is where I am having trouble. What makes up the built itself? There is a SoX linux executable in /usr/local/binwhich is a single file, but there are also a ton more files there that seem all related to making SoX and its dependencies work. Here is a list of files within /usr/local/bin on the working build I have.

我试图通过FTP导出所有这些文件，然后将它们导入到另一个干净的EC2实例中，但是即使在运行export PATH=$PATH:/usr/local/bin之后，由于依赖关系问题，SoX也将无法运行.显然，仅导出这些文件是不够的.

I attempted to export all these files via FTP and then import them into another clean EC2 instance, but even after running export PATH=$PATH:/usr/local/bin SoX would fail to run because of a dependency issue. It is clear that just exporting these files is not enough.

• 如何导出和导出什么内容以使带有MP3支持的SoX的工作版本脱离EC2实例，并使其包含在我的AWS Lambda函数中?
• 要使MP3正常运行，是否需要很长的已安装依赖项列表?
• 除了需要包含的/usr/local/bin目录中的文件以外，还有其他文件吗?

• How do I export and what do I export to take my working build of SoX with MP3 support off the EC2 instance and include it to work within my AWS Lambda Function?
• Is the long list of installed dependencies to get MP3 to work going to make this impossible?
• Are there files in more than just the /usr/local/bin directory that I need to include?

我真的不知道该怎么办.请帮助:(

I really don't know where else to go with this. Please help :(

推荐答案

花了我3个月的时间才最终发布，我在同一天就知道了.

Took me 3 months to finally post, and I figure it out the same day.

经过更多研究，我得出的结论是，我需要创建一个SoX的静态版本，并带有转换MP3所需的依赖项.我最终整理了一个脚本，以基于此博客创建静态构建帖子和此博客文章.

After doing some more research I came to the conclusion that I needed to create a static build of SoX with the necessary dependencies to convert MP3. I ended up putting together a script to create a static build based on this blog post, and this blog post.

我将两者结合的原因是因为我使用的是Amazons Linux AMI，这是非常准系统.因此，第二个博客中提到的所有debian和apt-get东西都不太可能.但是，我需要确保安装了编译器，这样才不会遇到任何错误.以下是我最终运行的内容.由于特权在EC2 Amazon Linux AMI实例上受到了相当大的限制，因此我以root身份完成了整个工作.作为普通用户，唯一需要运行的是最后的PATH导出.

The reason I combined a bit from both was because I was using Amazons Linux AMI, which is very barebones. So all the debian and apt-get stuff mentioned in the second blog would not likely work. However, I needed to ensure I had a complier installed so I wouldn't run into any errors. Below is what I ended up running. I did the whole thing as root, since privileges are fairly restricted on EC2 Amazon Linux AMI instances. The only thing that needs to be ran as the regular user is the PATH export at the very end.

sudo yum update
sudo yum install gcc44 gcc-c++ libgcc44 cmake –y

# now grab sox and its dependencies
mkdir -p deps
mkdir -p deps/unpacked
mkdir -p deps/built
mkdir -p deps/built/libmad
mkdir -p deps/built/sox
mkdir -p deps/built/lame
wget -O deps/sox-14.4.1.tar.bz2 "http://downloads.sourceforge.net/project/sox/sox/14.4.1/sox-14.4.1.tar.bz2?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fsox%2Ffiles%2Fsox%2F14.4.1%2F&ts=1416316415&use_mirror=heanet"
wget -O deps/libmad-0.15.1b.tar.gz "http://downloads.sourceforge.net/project/mad/libmad/0.15.1b/libmad-0.15.1b.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fmad%2Ffiles%2Flibmad%2F0.15.1b%2F&ts=1416316482&use_mirror=heanet"
wget -O deps/lame-3.99.5.tar.gz "http://downloads.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Flame%2Ffiles%2Flame%2F3.99%2F&ts=1416316457&use_mirror=kent"

# unpack the dependencies
pushd deps/unpacked
tar xvfpj ../sox-14.4.1.tar.bz2
tar xvfpz ../libmad-0.15.1b.tar.gz
tar xvfpz ../lame-3.99.5.tar.gz
popd

# build libmad, statically
pushd deps/unpacked/libmad-0.15.1b
./configure --disable-shared --enable-static --prefix=$(realpath ../../built/libmad)
# Patch makefile to remove -fforce-mem
sed s/-fforce-mem//g < Makefile > Makefile.patched
cp Makefile.patched Makefile
make
make install
popd

# build lame, statically
pushd deps/unpacked/lame-3.99.5
./configure --disable-shared --enable-static --prefix=$(realpath ../../built/lame)
make
make install
popd

# build sox, statically
pushd deps/unpacked/sox-14.4.1
./configure --disable-shared --enable-static --prefix=$(realpath ../../built/sox) \
    LDFLAGS="-L$(realpath ../../built/libmad/lib) -L$(realpath ../../built/lame/lib)" \
    CPPFLAGS="-I$(realpath ../../built/libmad/include) -I$(realpath ../../built/lame/include)" \
    --with-mad --with-lame --without-oggvorbis --without-oss --without-sndfile --without-flac  --without-gomp
make -s
make install
popd

cp deps/built/sox/bin/sox .
rm -rf deps/built
rm -rf deps/unpacked

此后，我以普通用户身份运行了以下

After this I ran the following as my normal user

export PATH=$PATH:/home/ec2-user

然后我可以跑袜！只需在终端中输入sox，即可在终端窗口中弹出命令列表.

I could then run sox! Just typing in sox in the terminal should bring up sox the command list in your terminal window.

更好的是，我能够下载单个sox linux可执行文件，然后将其上传到全新的EC2实例.我必须在新实例上运行才能使其正常运行，唯一的事情如下:

Even better, I was able to then download the single sox linux executable file, and then upload it to a brand new EC2 instance. The only things I had to run on the new instance to get it to work was the following:

sudo yum update
sudo yum install gcc44 gcc-c++ libgcc44 cmake –y
export PATH=$PATH:/home/ec2-user

我担心我必须尝试重新构建它而不进行yum更新或yum安装，因为我最终希望将其放入Lambda函数中，但这不是必须的！当我将其添加到测试Lambda函数并上传时，根据云日志，它可以正常工作.这必须意味着Lambda容器中使用的Amazon Linux AMI已经安装了yum更新和正确的编译器.

I was worried that I would have to try and build it again without making the yum update or yum installs, since I ultimately want to put this in a Lambda function, but that was not necessary! When I added it to my test Lambda function and uploaded, it worked without an issue according to the cloud logs. That must mean the Amazon Linux AMI used within Lambda containers already has the yum updates and proper compilers installed.

这篇关于如何在不受AWS的Linux AMI限制的情况下，将SoX二进制文件与对NodeJS AWS Lambda函数的MP3支持一起打包?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！