Perl 守护进程与子守护进程 [英] Perl daemonize with child daemons

问题描述

我必须在我的代码中使用守护进程.我需要一个控制守护进程，它不断检查数据库中的任务并监督子守护进程.控制守护进程必须将任务分配给子守护进程、控制任务、如果其中一个死亡则创建新的子进程等.子守护进程为它们检查数据库中的任务(通过 PID).为此，我应该如何实现守护进程?

I have to employ daemons in my code. I need a control daemon that constantly checks the database for the tasks and supervises child daemons. The control daemon must assign tasks to the child daemons, control tasks, create new children if one of them dies, etc. The child daemons check database for tasks for them (by PID). How should I implement daemons for this purpose?

推荐答案

Daemon 只是后台进程长时间运行"的代名词.所以答案是视情况而定".Perl 有两种主要的多处理方式:

Daemon is just a code word for "background process that runs a long time". So the answer is 'it depends'. Perl has two major ways of doing multiprocessing:

您将子例程作为线程运行，与主程序代码并行运行.(然后可能只监视线程状态).

You run a subroutine as a thread, in parallel with the main program code. (Which may then just monitor thread states).

创建线程的开销更高，但它更适合共享内存"风格的多处理，例如当您来回传递大量数据时.有几个库可以使线程之间的信息传递变得非常简单.我个人非常喜欢 Thread::Queue, Thread::Semaphore 和 Storable.

The overhead of creating a thread is higher, but it's better suited for 'shared memory' style multiprocessing, e.g. when you're passing significant quantities of data back and forth. There's several libraries that make passing information between threads positively straightforward. Personally I quite like Thread::Queue, Thread::Semaphore and Storable.

特别是 - Storable 有 freeze 和 thaw 可以让你在队列中移动复杂的数据结构(例如对象/散列)，非常有用.

In particular - Storable has freeze and thaw which lets you move complex data structures (e.g. objects/hashes) around in queues, which is very useful.

基本线程示例:

#!/usr/bin/perl

use strict;
use warnings;

use threads;

use Thread::Queue;

my $nthreads = 5;

my $process_q = Thread::Queue->new();
my $failed_q  = Thread::Queue->new();

#this is a subroutine, but that runs 'as a thread'.
#when it starts, it inherits the program state 'as is'. E.g.
#the variable declarations above all apply - but changes to
#values within the program are 'thread local' unless the
#variable is defined as 'shared'.
#Behind the scenes - Thread::Queue are 'shared' arrays.

sub worker {
    #NB - this will sit a loop indefinitely, until you close the queue.
    #using $process_q -> end
    #we do this once we've queued all the things we want to process
    #and the sub completes and exits neatly.
    #however if you _don't_ end it, this will sit waiting forever.
    while ( my $server = $process_q->dequeue() ) {
        chomp($server);
        print threads->self()->tid() . ": pinging $server
";
        my $result = `/bin/ping -c 1 $server`;
        if ($?) { $failed_q->enqueue($server) }
        print $result;
    }
}

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

#we 'end' process_q  - when we do, no more items may be inserted,
#and 'dequeue' returns 'undefined' when the queue is emptied.
#this means our worker threads (in their 'while' loop) will then exit.
$process_q->end();

#start some threads
for ( 1 .. $nthreads ) {
    threads->create( &worker );
}

#Wait for threads to all finish processing.
foreach my $thr ( threads->list() ) {
    $thr->join();
}

#collate results. ('synchronise' operation)
while ( my $server = $failed_q->dequeue_nb() ) {
    print "$server failed to ping
";
}

可储存

说到可存储，我认为值得单独举一个例子，因为它可以方便地移动数据.

Storable

When it comes to Storable, this is worth a separate example I think, because it's handy to move data around.

use Storable qw ( freeze thaw );
use MyObject;    #home made object.
use Thread::Queue;

my $work_q = Thread::Queue->new();

sub worker_thread {
    while ( my $packed_item = $work_q->dequeue ) {
        my $object = thaw($packed_item);
        $object->run_some_methods();
        $object->set_status("processed");
        #maybe return $object via 'freeze' and a queue?
    }
}

my $thr       = threads->create( &worker_thread );
my $newobject = MyObject->new("some_parameters");
$work_q->enqueue( freeze($newobject) );
$work_q->end();
$thr->join();

因为您在队列内传递对象，所以您在线程之间有效地克隆了它.所以请记住，一旦您对它的内部状态做了一些事情，您可能需要冻结它并以某种方式返回"它.但这确实意味着您可以异步执行此操作，而无需仲裁锁定或共享内存.您可能还会发现能够存储"和检索"和对象很有用 - 这可以按您的预期工作.(尽管我敢说，如果您要检索存储的对象，您可能需要注意模块版本与定义属性的可用性)

Because you're passing the object around within the queue, you're effectively cloning it between threads. So bear in mind that you may need to freeze it and 'return' it somehow once you've done something to it's internal state. But it does mean you can do this asynchronously without needing to arbitrate locking or shared memory. You may also find it useful to be able to 'store' and 'retrieve' and object - this works as you might expect. (Although I daresay you might need to be careful about availability of module versions vs. defined attributes if you're retrieving a stored object)

您的脚本会自我克隆，留下一个父母"和孩子"——然后孩子通常会发散并做一些不同的事情.这使用了内置 fork() 的 Unix，因此它得到了很好的优化并且通常非常高效 - 但因为它的级别很低，意味着很难进行大量数据传输.你最终会做一些稍微复杂的事情来进行进程间通信 - IPC.(有关详细信息，请参阅 perlipc).它很有效，尤其是因为大多数 fork() 实现都会进行惰性数据复制 - 您的进程的内存空间仅在需要时才分配，例如当它改变时.

Your script clones itself, leaving a 'parent' and 'child' - the child then generally diverges and does something different. This uses the Unix built in fork() which as a result is well optimised and generally very efficient - but because it's low level, means it's difficult to do lots of data transfer. You'll end up some slightly complicated things to do Interprocess communication - IPC. (See perlipc for more detail). It's efficient not least because most fork() implementations do a lazy data copy - memory space for your process is only allocated as it's needed e.g. when it's changed.

因此，如果您想委派大量不需要父母太多监督的任务，这真的很好.例如 - 您可能 fork 一个网络服务器，因为孩子正在读取文件并将它们传送到特定的客户端，而父母并不在乎.或者，如果您想花费大量 CPU 时间来计算结果，并且只将结果传回，您可能会这样做.

It's therefore really good if you want to delegate a lot of tasks that don't require much supervision from the parent. For example - you might fork a web server, because the child is reading files and delivering them to a particular client, and the parent doesn't much care. Or perhaps you would do this if you want to spend a lot of CPU time computing a result, and only pass that result back.

Windows 也不支持它.

It's also not supported on Windows.

有用的库包括 Parallel::ForkManager

Useful libraries include Parallel::ForkManager

分叉"代码的基本示例看起来有点像这样:

A basic example of 'forking' code looks a bit like this:

#!/usr/bin/perl
use strict;
use warnings;
use Parallel::ForkManager;

my $concurrent_fork_limit = 4;

my $fork_manager = Parallel::ForkManager->new($concurrent_fork_limit);

foreach my $thing ( "fork", "spoon", "knife", "plate" ) {
    my $pid = $fork_manager->start;
    if ($pid) {
        print "$$: Fork made a child with pid $pid
";
    } else {
        print "$$: child process started, with a key of $thing ($pid)
";
    }
    $fork_manager->finish;
}

$fork_manager->wait_all_children();

哪个适合您?

因此，如果没有更多关于您要实现的目标的详细信息，就很难说.这就是为什么 StacKOverflow 通常喜欢展示一些工作，你尝试过的方法等.

Which is right for you?

So it's hard to say without a bit more detail about what you're trying to accomplish. This is why StacKOverflow usually likes to show some working, approaches you've tried, etc.

我通常会说:

• 如果您需要传递数据，请使用线程.Thread::Queue 尤其是与 Storable 结合使用时非常适合它.

• if you need to pass data around, use threads. Thread::Queue especially when combined with Storable is very good for it.

如果不这样做，fork(在 Unix 上)通常会更快/更高效.(但光靠速度通常是不够的——先写容易理解的东西，然后再追求速度.通常没什么关系).

if you don't, forks (on Unix) are generally faster/more efficient. (But fast alone isn't usually enough - write understandable stuff first, and aim for speed second. It doesn't matter much usually).

尽可能避免产生过多线程 - 它们在内存和创建开销上相当密集.在工作线程"编程风格中使用固定数量比重复创建新的、短暂的线程要好得多.(另一方面 - fork 实际上非常擅长这一点，因为它们不会复制您的整个过程).

Avoid where possible spawning too many threads - they're fairly intensive on memory and creation overhead. You're far better off using a fixed number in a 'worker thread' style of programming, than repeatedly creating new, short lived threads. (On the other hand - forks are actually very good at this, because they don't copy your whole process).

我建议在您提供的场景中 - 您正在查看线程和队列.您的父进程可以通过 threads -> 跟踪子线程.list() 和 join 或 create 以保持正确的数字.并且可以通过中央队列将数据提供给您的工作线程.或者有多个队列 - 每个孩子"一个，并将其用作任务分配系统.

I would suggest in the scenario you give - you're looking at threads and queues. Your parent process can track child threads via threads -> list() and join or create to keep the right number. And can feed data via a central queue to your worker threads. Or have multiple queues - one per 'child' and use that as a task assignment system.

这篇关于Perl 守护进程与子守护进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！