惯用的方式来运送Erlang编写的命令行工具 [英] Idiomatic way to ship command line tools written in Erlang

查看：140
本文介绍了惯用的方式来运送Erlang编写的命令行工具的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

问题



大多数关于Erlang的文章和书籍我都可以专注于创建长时间运行的类似服务器的应用程序，从而使命令行工具创建过程不被覆盖



我有一个包含3个应用程序的多应用程序rebar3项目：




  • myweb - 一个牛仔基于Web服务;

  • mycli - 用于为 myweb ;

  • mylib - 由 myweb 和 mycli 使用的库取决于NIF。



作为构建的结果，我想要获得这样的工件：


  1. 一个用于提供http请求的Web部件的可执行文件;

  2. 用于资产准备的可执行命令行工具;

  3. 上述使用的一组库。



要求




  • cli应该行为e喜欢一个理智的非交互式命令行工具：处理参数，处理stdin / stdout，返回非零退出代码错误等;

  • 服务器和cli都应该能够使用NIFs;

  • 应该很容易将工件打包成一组deb / rpm包，所以服务器和cli都应该重用常见的依赖关系。

    • < $>

    事情尝试到目前为止



    构建一个escript



    我在野外看到的一种方法是创建一个自包含的escript文件。至少 rebar 和 relx 这样做。所以我试了一下。



    优点：




    • 支持命令行参数;

    • 如果错误返回非零退出代码。



    缺点：




    • 将所有依赖项嵌入到单个文件中， code> mylib ;

    • 自 *。所以文件嵌入到生成的escript文件它们不能在运行时加载，因此NIF不起作用（请参阅 erlang rebar escriptize& nifs ）;

    • rebar3 escriptize 不会很好地处理依赖关系（请参阅 bug 1139 ）。



    未知数： / p>


    • 应该是cli应用程序成为一个正确的OTP应用程序;

    • 应该有一个监督树; / li>
    • shoul如果是这样，那么在处理资产时如何阻止它？



    构建一个版本



    另外一种构建命令行工具的方法在我如何开始：Erlang 文章Fred Hebert。



    优点：




    • 每个依赖关系应用程序都进入自己的目录，方便您分享和打包它们。



    缺点：




    • 没有定义的入口点例如escript的 main / 1 ;

    • ，因此必须手动处理命令行参数和退出代码。



    未知数




    • 以非互动方式对cli OTP应用进行建模;

    • 如何处理资产时如何停止应用？





    上述两种方法都不适合我。



    它将从两个世界得到最好的：获取由escript提供的基础设施，例如 main / 1 入口点，命令行参数和退出代码处理，同时仍然有一个很好的目录结构，易于打包，并且不妨碍使用NIF。

    解决方案

    无论如果你是在Erlang中启动一个长时间运行的守护进程的应用程序，或者是一个CLI命令，你总是需要以下几点：


    1. erts 应用程序 - 特定版本中的VM和内核

    2. Erlang OTP应用程序

    3. 您的应用程序的依赖项

    5. CLI入口点

    然后在任一情况下，CLI入口点必须启动Erlang VM，执行它应该在给定的情况下执行的代码。然后它将退出或继续运行 - 长时间运行的应用程序的后期。



    CLI入口点可以是启动Erlang VM的任何东西，例如 escript 脚本， sh ， bash 等在通用shell中的 escript 的明显优势是，已经在Erlang VM的上下文中执行 escript ，所以没有需要处理启动/停止虚拟机。



    您可以通过两种方式启动Erlang VM：


    1. 使用系统范围的Erlang VM

    2. 使用嵌入式Erlang 发布

    在第一种情况下，您不提供 erts 或任何OTP应用程序与您的包，您只使一个特定的Erlang版本为您的应用程序的依赖。在第二种情况下，您提供 erts 和所有必需的OTP应用程序以及应用程序的依赖关系。



    第二种情况下，您还需要在启动时正确设置代码根虚拟机但这很容易，请参阅Erlang用于启动系统范围的VM的 erl 脚本：

     ＃位置：/ usr / local / lib / erlang / bin / erl 
     ROOTDIR =/ usr / local / lib / erlang
     BINDIR = $ ROOTDIR / 7.2.1 / bin 
     EMU = beam 
     PROGNAME =`echo $ 0 | sed的/.* \ ///'`
     export EMU 
     export ROOTDIR 
     export BINDIR 
     export PROGNAME 
     exec$ BINDIR / erlexec$ { 1 +$ @}

    这可以由脚本处理，例如 node_package 工具系统。我正在使用我自己的构建工具我自己的fork a href =https://github.com/yoonka/builderl =nofollow> builderl 。我只是说，所以你知道，如果我设法自定义，你也将能够做到这一点：）



    一旦Erlang VM启动，您的应用程序应该能够加载和启动任何应用程序，由Erlang提供或与您的应用程序一起提供（包括您提到的 mylib 库）。以下是一些例子：



    escript 示例



    请参阅这个 builderl.esh 示例我如何处理从 builderl 加载其他Erlang应用程序。 escript 脚本假定Erlang安装是相对于执行它的文件夹。当它是另一个应用程序的一部分时，例如 humbundee ， load_builderl .hrl 包含文件编译并加载 bld_load ，而后者又将所有剩余的模块加载到 bld_load：boot / 3 。注意如何使用标准的OTP应用程序而不指定它们在哪里 - builderl 正在执行 escript 等等应用程序从我们的系统安装的位置加载（ / usr / local / lib / erlang / lib / ）。如果您的应用程序使用的库，例如 mylib ，安装在其他地方，所有您需要做的是将该位置添加到Erlang路径，例如 代码：add_path 。 Erlang将自动将代码中使用的模块加载到添加到代码路径列表的文件夹中。



    嵌入式Erlang



    但是，如果应用程序是独立于系统范围的Erlang安装安装的正确的OTP版本，则会相同。这是因为在这种情况下，该脚本由属于该嵌入式Erlang版本的 escript 执行，而不是系统范围版本（即使已安装）。所以它知道属于该版本的所有应用程序的位置（包括您的应用程序）。例如 riak 确实如此 - 在他们的包中，他们提供了一个 erts 和所有依赖的Erlang应用程序的嵌入式Erlang版本。这样可以启动 riak ，而不会在主机操作系统上安装Erlang。这是FreeBSD上的 riak 包的摘录：

     ％tar -tf riak2-2.1.1_1.txz 
     / usr / local / sbin / riak 
     /usr/local/lib/riak/releases/start_erl.data 
     / usr / local / lib / riak / releases / 2.1.0 / riak.rel 
     / usr / local / lib / riak / releases / RELEASES 
     /usr/local/lib/riak/erts-5.10.3/bin/erl 
     /usr/local/lib/riak/erts-5.10.3/bin/beam 
     /usr/local/lib/riak/erts-5.10.3/bin/erlc 
     / usr /本地/ lib / riak / lib / stdlib-1.19.3 / ebin / re.beam 
     /usr/local/lib/riak/lib/ssl-5.3.1/ebin/tls_v1.beam 
     / usr / local / lib / riak / lib / crypto-3.1 / ebin / crypto.beam 
     /usr/local/lib/riak/lib/inets-5.9.6/ebin/inets.beam 
     / usr / local / lib / riak / lib / bitcask-1.7.0 / ebin / bitcask.app 
     /usr/local/lib/riak/lib/bitcask-1.7.0/ebin/bitcask.beam 
    （...）

    sh / bash



    原则上与上述不同，显式地调用你所用的函数在启动Erlang VM（入口点或 main 函数时调用它）执行。



    考虑这个脚本， builderl 生成以启动Erlang应用程序只是为了执行指定的任务（生成 RELEASES 文件），之后节点关闭：

     ＃！/ bin / sh 
     START_ERL =`cat releases / start_erl.data` 
     APP_VSN = $ {START_ERL＃*} 
     run_erl -daemon ../hbd/shell/ ../hbd/logexec erl ../hbd发行版/ start_erl.data -config releases / $ APP_VSN / hbd.config -args_file ../hbd/etc/vm.args -boot releases / $ APP_VSN / humbundee -noshell -noinput -eval \ {ok，Cwd} = file：get_cwd（），release_handler：create_RELEASES（Cwd，\\\releases\\\\，\\\\/ $ APP_VSN / humbundee.rel\\ \\\\，[]），init：stop（）\

    是一个类似的脚本，但不启动任何特定的代码或应用程序。相反，它会启动一个正确的OTP版本，因此哪些应用程序是启动的，按照什么顺序取决于版本（由 -boot 选项指定）。

     ＃！/ bin / sh 
     START_ERL =`cat releases / start_erl.data` 
     APP_VSN = $ {START_ERL＃*} 
     run_erl -daemon ../hbd/shell/ ../hbd/logexec erl ../hbd发行版本/ start_erl.data -config releases / $ APP_VSN / hbd.config -args_file ../hbd/ etc / vm.args -boot releases / $ APP_VSN / humbundee

    在 vm.args 文件，如果需要，您可以为应用程序提供其他路径，例如：

      pa lib / humbundee / ebin lib / yolf / ebin deps / goldrush / ebin deps / lager / ebin deps / yajler / ebin

    在这个例子中，这些是相对的，但是如果您的应用程序安装到标准的知名位置，则可以是绝对的。另外，只有当您使用系统范围的Erlang安装并且需要添加额外的路径才能找到Erlang应用程序，或者如果您的Erlang应用程序位于非标准位置（例如不在 lib 文件夹，如Erlang OTP所要求）。在适当的嵌入式Erlang版本中，应用程序位于代码根/ lib 文件夹，Erlang能够加载这些应用程序，而不指定任何其他路径。



    求和以及其他注意事项



    Erlang应用程序的部署与使用脚本语言编写的其他项目没有太大差异，例如红宝石或python项目。所有这些项目都必须处理类似的问题，我相信每个操作系统的包裹管理都是以这些方式处理的：


    1. 了解您的操作系统如何处理具有运行时依赖性的打包项目。


    2. 查看如何为您的操作系统打包其他Erlang应用程序，大部分通常由所有主要系统分发：RabbitMQ，Ejabberd，Riak等。只需下载包并将其解压缩到一个文件夹，那么您将看到所有文件的放置位置。


    编辑 - 参考要求



    回到您的要求，您有以下选择：


    1. 将Erlang作为OTP版本系统安装，作为嵌入式Erlang，或作为包含某些随机文件夹中的应用程序的袋子（对不起Rebar）


    2. 您可以以 sh 或 escript 脚本的形式提供多个入口点从安装的版本执行应用程序的选择。只要您配置代码根和路径到这些应用程序正确（如上所述），两者都可以工作。


    然后，您的每个应用程序： myweb 和 mycli ，将需要在其自己的新上下文中执行，例如启动一个新的VM实例并执行所需的应用程序（来自相同的Erlang版本）。在 myweb 的情况下，入口点可以是 sh 脚本，根据发布启动新节点（类似于了Riak）。在 mycli 的情况下，入口点可以是 escript ，一旦任务完成，该进程完成执行。



    但是，即使从 sh 开始，也可以创建一个退出VM的短期运行任务 - 请参见示例以上。在这种情况下， mycli 将需要单独的发行文件 - 脚本和启动引导虚拟机。当然，也可以从 escript 启动长时间运行的Erlang VM。



    我提供了一个示例项目它一次使用所有这些方法， humbundee 。一旦它被编译，它提供了三个访问点：


    1. cmd li>
    2. humbundee 发行。

    3. builder.esh escript 。

    第一个用于启动节点进行安装，然后关闭它。第二个用于启动长时间运行的Erlang应用程序。第三个是安装/配置节点的构建工具。一旦发布已经创建，项目就是这样的：

      $：〜/ work / humbundee / tmp / rel％ ls | tr\\\

     bin 
     erts-7.3 
     etc 
     lib 
    发布
     
     $：〜/ work / humbundee / tmp / rel％ls bin | tr\\\

     builderl.esh 
     cmd.boot 
     humbundee.boot 
     epmd 
     erl 
     escript 
     run_erl 
     to_erl 
    （...）
     
     $：〜/ work / humbundee / tmp / rel％ls lib | tr\\\

     builderl-0.2.7 
     compiler-6.0.3 
     deploy-0.0.1 
     goldrush-0.1.7 
     humbundee- 0.0.1 
     kernel-4.2 
     lager-3.0.1 
     mnesia-4.13.3 
     sasl-2.7 
     stdlib-2.8 
     syntax_tools-1.7 
     yajler-0.0.1 
     yolf-0.1.1 
     
     $：〜/ work / humbundee / tmp / rel％ls releases / hbd-0.0.1 | tr\\\

     builderl.config 
     cmd.boot 
     cmd.rel 
     cmd.script 
     humbundee.boot 
     humbundee.rel 
     humbundee.script 
     sys.config.src

    cmd 入门点将使用应用程序 deploy-0.0.1 和 builderl-0.2.7 以及发行文件 cmd.boot ， cmd.script 和一些OTP应用程序。标准的 humbundee 入门点将使用除 builderl 和 deploy 。然后， builderl.esh escript将使用应用程序 deploy-0.0.1 和 builderl-0.2 0.7 。全部来自同一嵌入式Erlang OTP安装。


    The problem

    Most of the articles and books about Erlang I could find focus on creating long running server-like applications leaving the process of command line tools creation not covered.

    I have a multi-app rebar3 project consisting of 3 applications:

    • myweb - a cowboy based web service;
    • mycli - a command line tool to prepare assets for myweb;
    • mylib - a library used by both myweb and mycli, depends on a NIF.

    As a result of the build I want to get such artifacts:

    1. an executable for the web part that is going to serve http requests;
    2. an executable command line tool for the assets preparation;
    3. a set of libraries used by the above.

    Requirements

    • cli should behave like a sane non interactive command line tool: handle arguments, deal with stdin/stdout, return non-zero exit code on error, etc;
    • both server and cli should be able to use NIFs;
    • it should be easy to package the artifacts as a set of deb/rpm packages, so both server and cli should reuse common dependencies.

    Things tried so far

    building an escript

    One of the ways I've seen in the wild is to create a self-contained escript file. At least rebar and relx do so. So I gave it a try.

    Pros:

    • has support for command line arguments;
    • in case of errors returns non-zero exit code.

    Cons:

    • embeds all the dependencies in a single file making it impossible to reuse mylib;
    • since *.so files get embedded into the resulting escript file they cannot be loaded at runtime, thus NIFs don't work (see erlang rebar escriptize & nifs);
    • rebar3 escriptize doesn't handle dependencies well (see bug 1139).

    Unknowns:

    • should the cli app become a proper OTP application;
    • should it have a supervision tree;
    • should it be started at all;
    • if so, how do I stop it when the assets have been processed?

    building a release

    Another way to build a command line tool was described in a How I start: Erlang article by Fred Hebert.

    Pros:

    • each of the dependency applications get into their own directory making it easy to share and package them.

    Cons:

    • there's no defined entry point like escript's main/1;
    • as a consequence both command line arguments and exit code must be handled manually.

    Unknowns:

    • how to model the cli OTP app in a non-interactive way;
    • how to stop the app when the assets have been processed?

    Neither of the approaches above seem to work for me.

    It would be get the best from both worlds: get the infrastructure that is provided by escript such as main/1 entry point, command line parameters and exit code handling while still having a nice directory structure that is easy to package and which doesn't hinder the use of NIFs.

    解决方案

    Regardless if you are starting a long-running daemon-like application in Erlang, or a CLI command, you always need the following:

    1. erts application - the VM and kernel in a particular version
    2. Erlang OTP applications
    3. Your applications' dependencies
    4. CLI entry point

    Then in either case the CLI entry point has to start the Erlang VM and execute the code that it supposed to execute in a given situation. Then it will either exit or continue running - the later for a long-running application.

    The CLI entry point can be anything that starts an Erlang VM, e.g. an escript script, sh, bash, etc. The obvious advantage of escript over generic shell is that escript is already being executed in the context of an Erlang VM, so no need to handle starting/stopping the VM.

    You can start Erlang VM in two ways:

    1. Use system-wide Erlang VM
    2. Use an embedded Erlang release

    In the first case you don't supply erts nor any OTP application with your package, you only make a particular Erlang version a dependency for your application. In the second case you supply erts and all required OTP applications along with your application's dependencies in your package.

    In the second case you also need to handle setting the code root properly when starting the VM. But this is quite easy, see the erl script that Erlang uses to start the system-wide VM:

    # location: /usr/local/lib/erlang/bin/erl
ROOTDIR="/usr/local/lib/erlang"
BINDIR=$ROOTDIR/erts-7.2.1/bin
EMU=beam
PROGNAME=`echo $0 | sed 's/.*\///'`
export EMU
export ROOTDIR
export BINDIR
export PROGNAME
exec "$BINDIR/erlexec" ${1+"$@"}

    This can be handled by scripts, for example the node_package tool that Basho uses to package their Riak database for all major operating systems. I am maintaining my own fork of it which I am using with my own build tool called builderl. I just say that so you know that if I managed to customize it you will well be able to do that as well :)

    Once the Erlang VM is started, your application should be able to load and start any application, either supplied with Erlang or with your application (and that includes the mylib library that you mentioned). Here are some examples how this could be achieved:

    escript example

    See this builderl.esh example how I handle loading other Erlang applications from builderl. That escript script assumes that the Erlang installation is relative to the folder from which it's executed. When it's a part of another application, like for example humbundee, the load_builderl.hrl include file compiles and loads bld_load, which in turn loads all remaining modules with bld_load:boot/3. Notice how I can use standard OTP applications without specifying where they are - builderl is being executed by escript and so all the applications are loaded from where they were installed (/usr/local/lib/erlang/lib/ on my system). If libraries used by your application, e.g. mylib, are installed somewhere else, all you need to do is add that location to the Erlang path, e.g. with code:add_path. Erlang will automatically load modules used in the code from folders added to the code path list.

    embedded Erlang

    However, the same would hold if the application was a proper OTP release installed independently from the system-wide Erlang installation. That's because in that case the script is executed by escript belonging to that embedded Erlang release rather than the system-wide version (even if it's installed). So it knows the location of all applications belonging to that release (including your applications). For example riak does exactly that - in their package they supply an embedded Erlang release that contains its own erts and all dependent Erlang applications. That way riak can be started without Erlang being even installed on the host operating system. This is an excerpt from a riak package on FreeBSD:

    % tar -tf riak2-2.1.1_1.txz
/usr/local/sbin/riak
/usr/local/lib/riak/releases/start_erl.data
/usr/local/lib/riak/releases/2.1.0/riak.rel
/usr/local/lib/riak/releases/RELEASES
/usr/local/lib/riak/erts-5.10.3/bin/erl
/usr/local/lib/riak/erts-5.10.3/bin/beam
/usr/local/lib/riak/erts-5.10.3/bin/erlc
/usr/local/lib/riak/lib/stdlib-1.19.3/ebin/re.beam
/usr/local/lib/riak/lib/ssl-5.3.1/ebin/tls_v1.beam
/usr/local/lib/riak/lib/crypto-3.1/ebin/crypto.beam
/usr/local/lib/riak/lib/inets-5.9.6/ebin/inets.beam
/usr/local/lib/riak/lib/bitcask-1.7.0/ebin/bitcask.app
/usr/local/lib/riak/lib/bitcask-1.7.0/ebin/bitcask.beam
(...)

    sh/bash

    This doesn't differ much in principle from the above apart from having to explicitly call the function that you want to execute when starting the Erlang VM (the entry point or the main function as you called it).

    Consider this script that builderl generates to start an Erlang application just to execute a specified task (generate the RELEASES file), after which the node shuts down:

    #!/bin/sh
START_ERL=`cat releases/start_erl.data`
APP_VSN=${START_ERL#* }
run_erl -daemon ../hbd/shell/ ../hbd/log "exec erl ../hbd releases releases/start_erl.data -config releases/$APP_VSN/hbd.config -args_file ../hbd/etc/vm.args -boot releases/$APP_VSN/humbundee -noshell -noinput -eval \"{ok, Cwd} = file:get_cwd(), release_handler:create_RELEASES(Cwd, \\\"releases\\\", \\\"releases/$APP_VSN/humbundee.rel\\\", []), init:stop()\""

    This is a similar script but doesn't start any specific code or application. Instead, it starts a proper OTP release, so which applications are started and in what order depends on the release (specified by the -boot option).

    #!/bin/sh
START_ERL=`cat releases/start_erl.data`
APP_VSN=${START_ERL#* }
run_erl -daemon ../hbd/shell/ ../hbd/log "exec erl ../hbd releases releases/start_erl.data -config releases/$APP_VSN/hbd.config -args_file ../hbd/etc/vm.args -boot releases/$APP_VSN/humbundee"

    In the vm.args file you can provide additional paths to your applications if required, e.g.:

    -pa lib/humbundee/ebin lib/yolf/ebin deps/goldrush/ebin deps/lager/ebin deps/yajler/ebin

    In this example these are relative, but could be absolute if your application is installed into a standard well-known location. Also, this would be only required if you are using the system-wide Erlang installation and need to add the additional paths to locate your Erlang applications, or if your Erlang applications are located in non-standard location (e.g. not in lib folder, as Erlang OTP requires). In a proper embedded Erlang release, where the applications are located in the code root/lib folder, Erlang is able to load those applications without specifying any additional paths.

    Summing up and other considerations

    The deployment of Erlang applications doesn't differ much from other projects written in scripting languages, e.g. ruby or python projects. All those projects have to deal with similar issues and I believe each operating system's package management deals with them one way or another:

    1. Get to know how your operating system deals with packaging projects that have run-time dependencies.

    2. See how other Erlang applications are packaged for your operating system, there are plenty of them that are usually distributed by all major systems: RabbitMQ, Ejabberd, Riak among others. Just download the package and unpack it to a folder, then you will see where all the files are placed.

    EDIT - reference the requirements

    Coming back to your requirements, you have the following choices:

    1. Install Erlang as an OTP release system-wide, as an embedded Erlang, or as a bag with applications in some random folders (sorry Rebar)

    2. You can have multiple entry points in the form of sh or escript scripts executing a selection of applications from the installed release. Both will work as long as you configured the code root and paths to those applications correctly (as outlined above).

    Then each of your applications: myweb and mycli, would need to be executed in its own new context, e.g. start a new VM instance and execute the required application (from the same Erlang release). In case of myweb the entry point can be a sh scripts that starts a new node according to the release (similar to Riak). In case of mycli the entry point can be an escript, which finishes executing once the task is completed.

    But it's entirely possible to create a short-running task that exits the VM even if it's started from sh - see the example above. In that case mycli would require separate release files - the script and boot to boot the VM. And of course it's also possible to start a long-running Erlang VM from escript.

    I provided an example project that uses all these methods at once, humbundee. Once it's compiled it provides three access points:

    1. The cmd release.
    2. The humbundee release.
    3. The builder.esh escript.

    The first one is used to start the node for installation and then shut it down. The second is used to start a long-running Erlang application. The third is a build tool to install/configure the node. This is how the project looks like once the release has been created:

    $:~/work/humbundee/tmp/rel % ls | tr " " "\n"
bin
erts-7.3
etc
lib
releases

$:~/work/humbundee/tmp/rel % ls bin | tr " " "\n"   
builderl.esh
cmd.boot
humbundee.boot
epmd
erl
escript
run_erl
to_erl
(...)

$:~/work/humbundee/tmp/rel % ls lib | tr " " "\n"
builderl-0.2.7
compiler-6.0.3
deploy-0.0.1
goldrush-0.1.7
humbundee-0.0.1
kernel-4.2
lager-3.0.1
mnesia-4.13.3
sasl-2.7
stdlib-2.8
syntax_tools-1.7
yajler-0.0.1
yolf-0.1.1

$:~/work/humbundee/tmp/rel % ls releases/hbd-0.0.1 | tr " " "\n"
builderl.config
cmd.boot
cmd.rel
cmd.script
humbundee.boot
humbundee.rel
humbundee.script
sys.config.src

    The cmd entry point will use application deploy-0.0.1 and builderl-0.2.7 as well as release files cmd.boot, cmd.script, and some OTP applications. The standard humbundee entry point will use all applications apart from builderl and deploy. Then the builderl.esh escript will use application deploy-0.0.1 and builderl-0.2.7. All from the same embedded Erlang OTP installation.

    这篇关于惯用的方式来运送Erlang编写的命令行工具的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
其他开发语言最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆