Perl用子守护程序守护 [英] Perl daemonize with child daemons

问题描述

我必须在代码中使用守护程序.我需要一个控制守护程序，该守护程序不断检查数据库中的任务并监督子守护程序.控制守护程序必须将任务分配给子守护程序，控制任务，如果其中一个子进程死亡，则创建新的子进程，等等.子守护程序检查数据库中是否有针对它们的任务(通过PID).为此，我应该如何实现守护程序?

I have to employ daemons in my code. I need a control daemon that constantly checks the database for the tasks and supervises child daemons. The control daemon must assign tasks to the child daemons, control tasks, create new children if one of them dies, etc. The child daemons check database for tasks for them (by PID). How should I implement daemons for this purpose?

推荐答案

Daemon只是长时间运行的后台进程"的代名词.因此，答案是取决于". Perl有两种​​主要的进行多处理的方式:

Daemon is just a code word for "background process that runs a long time". So the answer is 'it depends'. Perl has two major ways of doing multiprocessing:

您可以将子例程作为线程与主程序代码并行运行. (然后可能只监视线程状态).

You run a subroutine as a thread, in parallel with the main program code. (Which may then just monitor thread states).

创建线程的开销较高，但更适合于共享内存"样式的多处理，例如当您来回传递大量数据时.有几个库使在线程之间传递信息非常简单.我个人非常喜欢 Thread::Queue ， Storable .

The overhead of creating a thread is higher, but it's better suited for 'shared memory' style multiprocessing, e.g. when you're passing significant quantities of data back and forth. There's several libraries that make passing information between threads positively straightforward. Personally I quite like Thread::Queue, Thread::Semaphore and Storable.

特别是-Storable具有freeze和thaw，可让您在队列中四处移动复杂的数据结构(例如，对象/哈希)，这非常有用.

In particular - Storable has freeze and thaw which lets you move complex data structures (e.g. objects/hashes) around in queues, which is very useful.

基本线程示例:

#!/usr/bin/perl

use strict;
use warnings;

use threads;

use Thread::Queue;

my $nthreads = 5;

my $process_q = Thread::Queue->new();
my $failed_q  = Thread::Queue->new();

#this is a subroutine, but that runs 'as a thread'.
#when it starts, it inherits the program state 'as is'. E.g.
#the variable declarations above all apply - but changes to
#values within the program are 'thread local' unless the
#variable is defined as 'shared'.
#Behind the scenes - Thread::Queue are 'shared' arrays.

sub worker {
    #NB - this will sit a loop indefinitely, until you close the queue.
    #using $process_q -> end
    #we do this once we've queued all the things we want to process
    #and the sub completes and exits neatly.
    #however if you _don't_ end it, this will sit waiting forever.
    while ( my $server = $process_q->dequeue() ) {
        chomp($server);
        print threads->self()->tid() . ": pinging $server\n";
        my $result = `/bin/ping -c 1 $server`;
        if ($?) { $failed_q->enqueue($server) }
        print $result;
    }
}

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

#we 'end' process_q  - when we do, no more items may be inserted,
#and 'dequeue' returns 'undefined' when the queue is emptied.
#this means our worker threads (in their 'while' loop) will then exit.
$process_q->end();

#start some threads
for ( 1 .. $nthreads ) {
    threads->create( \&worker );
}

#Wait for threads to all finish processing.
foreach my $thr ( threads->list() ) {
    $thr->join();
}

#collate results. ('synchronise' operation)
while ( my $server = $failed_q->dequeue_nb() ) {
    print "$server failed to ping\n";
}

可存储

在谈到Storable时，我认为这值得一个单独的示例，因为它便于移动数据.

Storable

When it comes to Storable, this is worth a separate example I think, because it's handy to move data around.

use Storable qw ( freeze thaw );
use MyObject;    #home made object.
use Thread::Queue;

my $work_q = Thread::Queue->new();

sub worker_thread {
    while ( my $packed_item = $work_q->dequeue ) {
        my $object = thaw($packed_item);
        $object->run_some_methods();
        $object->set_status("processed");
        #maybe return $object via 'freeze' and a queue?
    }
}

my $thr       = threads->create( \&worker_thread );
my $newobject = MyObject->new("some_parameters");
$work_q->enqueue( freeze($newobject) );
$work_q->end();
$thr->join();

因为要在队列中传递对象，所以实际上是在线程之间克隆对象.因此请记住，一旦对它的内部状态做了一些处理，您可能需要冻结它并以某种方式返回"它.但这确实意味着您可以异步执行此操作，而无需仲裁锁定或共享内存.您可能还会发现能够存储"，检索"和对象很有用-这可以按您期望的那样工作. (尽管我敢说，如果要检索存储的对象，可能需要注意模块版本和定义的属性的可用性)

Because you're passing the object around within the queue, you're effectively cloning it between threads. So bear in mind that you may need to freeze it and 'return' it somehow once you've done something to it's internal state. But it does mean you can do this asynchronously without needing to arbitrate locking or shared memory. You may also find it useful to be able to 'store' and 'retrieve' and object - this works as you might expect. (Although I daresay you might need to be careful about availability of module versions vs. defined attributes if you're retrieving a stored object)

您的脚本会克隆自己，留下一个父"和子"，然后该子通常会发散并执行其他操作.它使用内置在fork()中的Unix，因此可以得到很好的优化，并且通常非常高效-但由于级别较低，因此很难进行大量的数据传输.您将完成一些稍微复杂的事情以进行进程间通信-IPC. (有关更多详细信息，请参见 perlipc ).这之所以有效是因为大多数fork()实现都会执行惰性数据复制-仅在需要时才分配进程的内存空间，例如当它改变时.

Your script clones itself, leaving a 'parent' and 'child' - the child then generally diverges and does something different. This uses the Unix built in fork() which as a result is well optimised and generally very efficient - but because it's low level, means it's difficult to do lots of data transfer. You'll end up some slightly complicated things to do Interprocess communication - IPC. (See perlipc for more detail). It's efficient not least because most fork() implementations do a lazy data copy - memory space for your process is only allocated as it's needed e.g. when it's changed.

因此，如果您要委派很多不需要父母太多监督的任务，那真的很好.例如-您可能fork一个Web服务器，因为孩子正在读取文件并将其传递给特定的客户端，而父级则不太在意.或者，如果您想花费大量的CPU时间来计算结果，而只将结果传回，则可能会这样做.

It's therefore really good if you want to delegate a lot of tasks that don't require much supervision from the parent. For example - you might fork a web server, because the child is reading files and delivering them to a particular client, and the parent doesn't much care. Or perhaps you would do this if you want to spend a lot of CPU time computing a result, and only pass that result back.

Windows也不支持.

It's also not supported on Windows.

有用的库包括 Parallel::ForkManager

Useful libraries include Parallel::ForkManager

分叉"代码的基本示例看起来像这样:

A basic example of 'forking' code looks a bit like this:

#!/usr/bin/perl
use strict;
use warnings;
use Parallel::ForkManager;

my $concurrent_fork_limit = 4;

my $fork_manager = Parallel::ForkManager->new($concurrent_fork_limit);

foreach my $thing ( "fork", "spoon", "knife", "plate" ) {
    my $pid = $fork_manager->start;
    if ($pid) {
        print "$$: Fork made a child with pid $pid\n";
    } else {
        print "$$: child process started, with a key of $thing ($pid)\n";
    }
    $fork_manager->finish;
}

$fork_manager->wait_all_children();

哪个最适合您?

因此，很难说出您要完成的目标的更多细节.这就是为什么StacKOverflow通常喜欢显示一些工作原理，您尝试过的方法等等.

Which is right for you?

So it's hard to say without a bit more detail about what you're trying to accomplish. This is why StacKOverflow usually likes to show some working, approaches you've tried, etc.

我通常会说:

• 如果需要传递数据，请使用线程. Thread::Queue特别适合与Storable结合使用.

• if you need to pass data around, use threads. Thread::Queue especially when combined with Storable is very good for it.

如果不这样做，则分叉(在Unix上)通常更快/更高效. (但是，仅靠速度通常是不够的-首先编写可理解的内容，然后以速度为目标.通常并不重要).

if you don't, forks (on Unix) are generally faster/more efficient. (But fast alone isn't usually enough - write understandable stuff first, and aim for speed second. It doesn't matter much usually).

避免在可能的情况下产生太多线程-它们在内存和创建开销上非常密集.与以重复方式创建短暂的新线程相比，以工作线程"编程风格使用固定数量要好得多. (另一方面，分叉实际上非常擅长此操作，因为它们不会复制您的整个过程).

Avoid where possible spawning too many threads - they're fairly intensive on memory and creation overhead. You're far better off using a fixed number in a 'worker thread' style of programming, than repeatedly creating new, short lived threads. (On the other hand - forks are actually very good at this, because they don't copy your whole process).

在您提供的方案中，我建议-您正在查看线程和队列.您的父进程可以通过threads -> list()和join或create跟踪子线程以保持正确的编号.并且可以通过中央队列将数据提供给您的工作线程.或者有多个队列-每个孩子"一个队列，并将其用作任务分配系统.

I would suggest in the scenario you give - you're looking at threads and queues. Your parent process can track child threads via threads -> list() and join or create to keep the right number. And can feed data via a central queue to your worker threads. Or have multiple queues - one per 'child' and use that as a task assignment system.

这篇关于Perl用子守护程序守护的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！