如何在Perl中限制并行线程的最大数量 [英] How to limit the max number of parallel threads in perl

问题描述

我有一个启动大量线程的程序(Perl)(每个线程负责基于数据处理创建图形). 我开始使用的每个线程:

I have a program (Perl) that kicks off a huge amount of threads (each one in charge of creating graphics based on data processing). Each thread I start using:

my @threads //list to store threads that have been launched

push @threads,threads->create(\mySubName,params...);

线程正确触发，但是过一会儿，在我打开了其中的几个Perl解释器后，它崩溃了(我认为它与内存有关?).因此，我的解决方案是限制每次打开的线程数(我选择了15个).我想在每个创建行之前添加一个子项，以检查是否可以在等待时触发下一个线程或执行睡眠一个完成.这就是我试图做到的方式.

The threads fire off correctly but after a while, after I have opened several of them Perl interpreter crashes (I assume it is related to memory?). So my solution is to limit the number of threads I open at a time, I picked 15. And I want to add a sub before each create line to check if it is ok to fire off the next thread or perform a sleep while I wait for one to finish. This is how I tried to do it.

sub checkThreads{
    my $addThread = 0;
    until($addThread){
        my $totalThreads = 0;
        foreach my $task (@threads){
            if($task->is_running()){$totalThreads++;}
        }
        if($totalThreads <= 15 ){
            print "Ok to add new thread, carry on!\n";
            $addthread = 1;
        }else{
            print "Waiting for $totalThreads threads to fire next one...\n";
            sleep 2;
        }
    }
}

所以每次我想创建一个新线程时，我都会调用

So each time I want to create a new thread I would just call

&checkThreads;

在我等待一些线程清理时，这会引起延迟. 问题是，当我调用该子程序时，我碰到了要检查的那一行:

And that would take care to create a delay while I wait for some threads to clean up. The problem is that when I call that sub, the moment I hit the line where I check:

$task->is_running()

程序退出并停止运行，没有任何错误或警告.我只想要一个对正在运行的线程进行计数的子程序来限制它们.

The program exits and stops running without any error or warning. I just want a sub that counts the running threads to limit them.

如何成功执行此计数?

我尝试过的其他方法正在评估以下行:

Other things I have tried are evaluating the following line:

scalar(threads->list());

但这给了我一个怪异的值，就像它是我认为是一个无可厚非的参考文献一样.

But that gives me a weird value, like it is an unblessed reference I believe that looks like:

threads=SCALAR(0x80fea8c)

推荐答案

Thread :: Semaphore 提供了计数信号量来限制并发:

my $sem = Thread::Semaphore->new(15); # max 15 threads
my @threads = map {
    # request a thread slot, waiting if none are available:
    $sem->down;
    threads->create(\&mySubName, @params)
} 0..100;
$_->join for @threads;

在您的函数中:

sub mySubName {
    do_stuff();
    # release slot:
    $sem->up;
}

这篇关于如何在Perl中限制并行线程的最大数量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！