制作一个运行24/7并从命名管道读取的Perl守护程序 [英] Making a Perl daemon that runs 24/7 and reads from named pipes

问题描述

我正在尝试使用perl制作日志分析器.分析仪将在AIX服务器上在后台运行24/7，并从syslog将日志定向到(来自整个网络)的管道中读取数据.基本上:

logs from network ----> named pipe A -------->   | perl daemon
                  ----> named pipe B -------->   | * reads pipes
                  ----> named pipe c -------->   | * decides what to do based on which pipe

例如，我希望我的守护程序能够配置为mail [e-mail]所有写入named pipe C的日志.为此，我假设守护程序需要具有一个哈希(对于perl来说是新的，但这似乎是一个适当的数据结构)，该哈希可以在运行中进行更改，并告诉它如何处理每个管道. /p>

这可能吗?还是应该在/etc中创建一个.conf文件来保存信息.像这样:

namedpipeA:'mail [e-mail]'
namedpipeB:save:'mail [e-mail]'

因此从A获取的任何内容都将被邮寄到[e-mail]，并且B的所有内容将被保存到日志文件中(通常是这样)，并将其发送到[e-mail]

这是我第一次使用Perl，而且是我第一次创建守护进程，因此无论如何我都必须遵守解决方案

我将介绍您的部分问题:如何编写一个处理IO的长期运行的Perl程序.

编写处理许多同时IO操作的Perl程序的最有效方法是使用事件循环.这将使我们能够为事件编写处理程序，例如在命名管道上出现一行"或电子邮件已成功发送"或我们收到了SIGINT".至关重要的是，它将允许我们在一个程序中组成任意数量的这些事件处理程序.这意味着您可以多任务"，但仍可以轻松地在任务之间共享状态.

我们将使用 AnyEvent 框架.它使我们能够编写事件处理程序(称为观察程序)，该处理程序可与Perl支持的任何事件循环一起使用.您可能不在乎使用哪个事件循环，因此此抽象可能对您的应用程序无关紧要.但是，这将使我们能够重用CPAN上可用的预先编写的事件处理程序； AnyEvent :: SMTP 处理电子邮件，

我们首先加载模块:

use strict;
use warnings;
use 5.010;
use AnyEvent;

然后，我们将创建一个时间监视程序或一个计时器":

my $t = AnyEvent->timer( after => 0, interval => 5, cb => sub {
    say "Hello";
});

请注意，我们将计时器分配给了变量.只要$t在范围内，这将使计时器保持活动状态.如果我们说undef $t，则计时器将被取消，并且永远不会调用回调.

关于回调，这是cb =>之后的sub { ... }，这就是我们处理事件的方式.事件发生时，将调用回调.我们做我们的事情，返回，事件循环根据需要继续调用其他回调.您可以在回调中做任何您想做的事情，包括取消和创建其他观察者.只是不要发出阻塞呼叫，例如system("/bin/sh long running process")或my $line = <$fh>或sleep 10.任何阻止操作的操作都必须由观察者完成；否则，事件循环将无法在等待该任务完成时运行其他处理程序.

现在我们有了一个计时器，我们只需要进入事件循环.通常，您将选择要使用的事件循环，并以事件循环文档中描述的特定方式输入事件循环. EV 是一个不错的选择，您可以通过调用EV::loop()进行输入.但是，我们将通过编写AnyEvent->condvar->recv让AnyEvent决定要使用的事件循环.不用担心这会做什么；这是一个习惯用法，意思是进入事件循环，永不返回". (在阅读有关AnyEvent的内容时，您会看到很多关于条件变量或condvars的信息.它们非常适合用于文档和单元测试中的示例，但您确实不想在程序中使用它们. '在文件中使用它们，您做错了什么.因此，假装它们暂时不存在，您将从一开始就编写非常干净的代码.这将使您前进许多CPAN作者！)

所以，仅出于完整性考虑:

AnyEvent->condvar->recv;

如果运行该程序，它将每五秒钟打印一次"Hello"，直到宇宙结束为止，或者更可能是使用控件c将其杀死.整洁的是，您可以在打印"Hello"之间的这五秒钟内执行其他操作，而只需添加更多观察者即可.

因此，现在开始从管道读取数据. AnyEvent通过其AnyEvent :: Handle模块使此操作非常容易. AnyEvent :: Handle可以连接到套接字或管道，并且只要有数据可读取，便会调用回调. (它也可以执行非阻塞写入，TLS和其他操作.但是我们现在不在乎.)

首先，我们需要打开一个管道:

use autodie 'open';
open my $fh, '<', '/path/to/pipe';

然后，我们用AnyEvent :: Handle包装它.创建Handle对象后，我们将在该管道上对其执行所有操作.您完全可以忘记$fh，AnyEvent :: Handle将直接处理它.

my $h = AnyEvent::Handle->new( fh => $fh );

现在，我们可以使用$h在管道可用时从管道中读取它们:

$h->push_read( line => sub {
    my ($h, $line, $eol) = @_;
    say "Got a line: $line";
});

这将调用在下一行可用时显示获得一行"的回调.如果要继续读取行，则需要使函数将自身推回读取队列，例如:

my $handle_line; $handle_line = sub {
    my ($h, $line, $eol) = @_;
    say "Got a line: $line";
    $h->push_read( line => $handle_line );
};
$h->push_read( line => $handle_line );

这将读取行并为每一行调用$handle_line->()，直到关闭文件.如果您想早点停止阅读，那很容易...在这种情况下，请不要再次push_read. (您不必在行级别进行读取；您可以要求在任何字节可用时调用回调.但这更加复杂，留给读者练习.)

因此，现在我们可以将所有这些内容绑定到一个守护程序中，该守护程序可以处理管道.我们想要做的是:为行创建一个处理程序，打开管道并处理这些行，最后设置一个信号处理程序以干净地退出程序.我建议对这种问题采取面向对象的方法.使用start和stop方法使每个动作(访问日志文件中的句柄")成为一个类，实例化一系列动作，设置信号处理程序以干净地停止这些动作，启动所有动作，然后进入事件循环.大量的代码与该问题并没有真正的关系，因此我们将做一些更简单的事情.但是在设计程序时请记住这一点.

#!/usr/bin/env perl
use strict;
use warnings;
use AnyEvent;
use AnyEvent::Handle;
use EV;

use autodie 'open';
use 5.010;

my @handles;

my $abort; $abort = AnyEvent->signal( signal => 'INT', cb => sub {
    say "Exiting.";
    $_->destroy for @handles;
    undef $abort; 
    # all watchers destroyed, event loop will return
});

my $handler; $handler = sub {
    my ($h, $line, $eol) = @_;
    my $name = $h->{name};
    say "$name: $line";
    $h->push_read( line => $handler );
};

for my $file (@ARGV) {
    open my $fh, '<', $file;
    my $h = AnyEvent::Handle->new( fh => $fh );
    $h->{name} = $file;
    $h->push_read( line => $handler );
}

EV::loop;

现在，您有了一个程序，该程序可以从任意数量的管道中读取一行，打印在任何管道上(以管道的路径为前缀)接收到的每一行，并在按Control-C时干净地退出！

I'm trying to make a log analyser using perl. The analyser would run 24/7 in the background on an AIX server and read from pipes that syslog directs logs to (from the entire network). Basically:

logs from network ----> named pipe A -------->   | perl daemon
                  ----> named pipe B -------->   | * reads pipes
                  ----> named pipe c -------->   | * decides what to do based on which pipe

So, for example, I want my daemon to be able to be configured to mail [e-mail] all logs that are written to named pipe C. For this, I'm assuming the daemon needs to have a hash (new to perl, but this seems like an appropriate data structure) that would be able to be changed on the fly and would tell it what to do with each pipe.

Is this possible? Or should I create a .conf file in /etc to hold the information. Something like this:

namedpipeA:'mail [e-mail]'
namedpipeB:save:'mail [e-mail]'

So getting anything from A will be mailed to [e-mail] and everything from B will be saved to a log file (like it is usually) AND it will be sent to [e-mail]

Seeing as this is my first time using Perl and my first time creating a daemon, is there anyway for me to make this while adhering to the KISS principal? Also, are there any conventions that I should stick to? If you could take into consideration my lack of knowledge when replying it would be most helpful.

解决方案

I'll cover part of your question: how to write a long-running Perl program that deals with IO.

The most efficient way to write a Perl program that handles many simultaneous IO operations is to use an event loop. This will allow us to write handlers for events, like "a line appeared on the named pipe" or "the email was sent successfully" or "we received SIGINT". Crucially, it will allow us to compose an arbitrary number of these event handlers in one program. This means that you can "multitask" but still easily share state between the tasks.

We'll use the AnyEvent framework. It lets us write event handlers, called watchers, that will work with any event loop that Perl supports. You probably don't care which event loop you use, so this abstraction probably doesn't matter to your application. But it will let us reuse pre-written event handlers available on CPAN; AnyEvent::SMTP to handle email, AnyEvent::Subprocess to interact with child processes, AnyEvent::Handle to deal with the pipes, and so on.

The basic structure of an AnyEvent-based daemon is very simple. You create some watchers, enter the event loop, and ... that's it; the event system does everything else. To get started, let's write a program that will print "Hello" every five seconds.

We start by loading modules:

use strict;
use warnings;
use 5.010;
use AnyEvent;

Then, we'll create a time watcher, or a "timer":

my $t = AnyEvent->timer( after => 0, interval => 5, cb => sub {
    say "Hello";
});

Note that we assign the timer to a variable. This keeps the timer alive as long as $t is in scope. If we said undef $t, then the timer would be cancelled and the callback would never be called.

About callbacks, that's the sub { ... } after cb =>, and that's how we handle events. When an event happens, the callback is invoked. We do our thing, return, and the event loop continues calling other callbacks as necessary. You can do anything you want in callbacks, including cancelling and creating other watchers. Just don't make a blocking call, like system("/bin/sh long running process") or my $line = <$fh> or sleep 10. Anything that blocks must be done by a watcher; otherwise, the event loop won't be able to run other handlers while waiting for that task to complete.

Now that we have a timer, we just need to enter the event loop. Typically, you'll choose an event loop that you want to use, and enter it in the specific way that the event loop's documentation describes. EV is a good one, and you enter it by calling EV::loop(). But, we'll let AnyEvent make the decision about what event loop to use, by writing AnyEvent->condvar->recv. Don't worry what this does; it's an idiom that means "enter the event loop and never return". (You'll see a lot about condition variables, or condvars, as you read about AnyEvent. They are nice for examples in the documentation and in unit tests, but you really don't want to ever use them in your program. If you're using them inside a .pm file, you're doing something very wrong. So just pretend they don't exist for now, and you'll write extremely clean code right from the start. And that'll put you ahead of many CPAN authors!)

So, just for completeness:

AnyEvent->condvar->recv;

If you run that program, it will print "Hello" every five seconds until the universe ends, or, more likely, you kill it with control c. What's neat about this is that you can do other things in those five seconds between printing "Hello", and you do it just by adding more watchers.

So, now onto reading from pipes. AnyEvent makes this very easy with its AnyEvent::Handle module. AnyEvent::Handle can connect to sockets or pipes and will call a callback whenever data is available to read from them. (It can also do non-blocking writes, TLS, and other stuff. But we don't care about that right now.)

First, we need to open a pipe:

use autodie 'open';
open my $fh, '<', '/path/to/pipe';

Then, we wrap it with an AnyEvent::Handle. After creating the Handle object, we'll use it for all operations on this pipe. You can completely forget about $fh, AnyEvent::Handle will handle touching it directly.

my $h = AnyEvent::Handle->new( fh => $fh );

Now we can use $h to read lines from the pipe when they become available:

$h->push_read( line => sub {
    my ($h, $line, $eol) = @_;
    say "Got a line: $line";
});

This will call the callback that prints "Got a line" when the next line becomes available. If you want to continue reading lines, then you need to make the function push itself back onto the read queue, like:

my $handle_line; $handle_line = sub {
    my ($h, $line, $eol) = @_;
    say "Got a line: $line";
    $h->push_read( line => $handle_line );
};
$h->push_read( line => $handle_line );

This will read lines and call $handle_line->() for each line until the file is closed. If you want to stop reading early, that's easy... just don't push_read again in that case. (You don't have to read at the line level; you can ask that your callback be called whenever any bytes become available. But that's more complicated and left as an exercise to the reader.)

So now we can tie this all together into a daemon that handles reading the pipes. What we want to do is: create a handler for lines, open the pipes and handle the lines, and finally set up a signal handler to cleanly exit the program. I recommend taking an OO approach to this problem; make each action ("handle lines from the access log file") a class with a start and stop method, instantiate a bunch of actions, setup a signal handler to cleanly stop the actions, start all the actions, and then enter the event loop. That's a lot of code that's not really related to this problem, so we'll do something simpler. But keep that in mind as you design your program.

#!/usr/bin/env perl
use strict;
use warnings;
use AnyEvent;
use AnyEvent::Handle;
use EV;

use autodie 'open';
use 5.010;

my @handles;

my $abort; $abort = AnyEvent->signal( signal => 'INT', cb => sub {
    say "Exiting.";
    $_->destroy for @handles;
    undef $abort; 
    # all watchers destroyed, event loop will return
});

my $handler; $handler = sub {
    my ($h, $line, $eol) = @_;
    my $name = $h->{name};
    say "$name: $line";
    $h->push_read( line => $handler );
};

for my $file (@ARGV) {
    open my $fh, '<', $file;
    my $h = AnyEvent::Handle->new( fh => $fh );
    $h->{name} = $file;
    $h->push_read( line => $handler );
}

EV::loop;

Now you have a program that reads a line from an arbitrary number of pipes, prints each line received on any pipe (prefixed with the path to the pipe), and exits cleanly when you press Control-C!

这篇关于制作一个运行24/7并从命名管道读取的Perl守护程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！