在任何给定时间运行 5 个脚本 [英] Have 5 scripts running at any given time

问题描述

我有一个启动 90 个不同 PHP 脚本的 bash 脚本(在 CentOS 6.4 下运行)，即.

#!/bin/bashphp path1/some_job_1.php&php path2/some_job_2.php&php path3/some_job_3.php&php path4/some_job_4.php&php path5/some_job_5.phpphp path6/some_job_6.php&php path7/some_job_7.php&php path8/some_job_8.php&php path9/some_job_9.php&php path10/some_job_10.php...出口 0

为了避免我的服务器超载，我使用 & 符号 &，它可以工作，但 我的目标是始终让 5 个脚本同时运行

有没有办法做到这一点?

解决方案

这个问题被弹出了好几次，但我找不到合适的答案.我想现在我找到了一个很好的解决方案！

不幸的是 parallel 不是标准发行版的一部分，而是 make 是.它有一个开关 -j 来做并行.

man make(1)]: (更多关于 make 并行执行的信息)

<块引用>

 -j [jobs], --jobs[=jobs]指定同时运行的作业(命令)的数量.如果-j 选项不止一个，最后一个有效.如果-j 选项不带参数给出，make 不会限制可以同时运行的作业数.

因此，使用适当的 Makefile 可以解决问题.

.PHONY: all $(PHP_DEST)# 创建一个目标数组，形式为 PHP1 PHP2 ... PHP90PHP_DEST := $(添加前缀 PHP, $(shell seq 1 1 90))# 默认目标全部:$(PHP_DEST)# 为每个目标运行正确的脚本$(PHP_DEST):N=$(subst PHP,,$@);php 路径$N/some_job_$N.php

它会创建 90 个 PHP# 目标，每个调用 php path#/some_job_#.php.如果您运行 make -j 5 那么它将运行 5 个 php 实例并行.如果一个完成，它会开始下一个.

我将 Makefile 重命名为 parallel.mak，我运行 chmod 700 parallel.mak 并添加了 #!/usr/bin/make -f 到第一行.现在可以调用 ./parallel.mak -j 5.

甚至你可以使用更复杂的 -l 开关:

<块引用>

 -l [load], --load-average[=load]指定不应该启动新的作业(命令)，如果有其他作业是否正在运行并且平均负载至少为负载(a浮点数).不带参数，删除先前的负载限制.

在这种情况下，make 将决定有多少可以根据系统负载启动作业.

我用 ./parallel.mak -j -l 1.0 对其进行了测试并且运行良好.它首先并行启动了 4 个程序

I have a bash script (running under CentOS 6.4) that launches 90 different PHP scripts, ie.

#!/bin/bash

php path1/some_job_1.php&
php path2/some_job_2.php&
php path3/some_job_3.php&
php path4/some_job_4.php&
php path5/some_job_5.php

php path6/some_job_6.php&
php path7/some_job_7.php&
php path8/some_job_8.php&
php path9/some_job_9.php&
php path10/some_job_10.php
...

exit 0

In order to avoid overloading my server, I use the ampersand &, it works, but my goal is to always have 5 scripts running at the same time

Is there a way to achieve this?

解决方案

This question is popped several times, but I could not find a proper answer for it. I think now I found a good solution!

Unfortunately parallel is not the part of the standard distributions, but make is. It has a switch -j to do makes parallel.

man make(1)]: (more info on make's parallel execution)

  -j [jobs], --jobs[=jobs]
        Specifies the number of jobs (commands) to run simultaneously.  If
        there  is  more than one -j option, the last one is effective.  If
        the -j option is given without an argument, make  will  not  limit
        the number of jobs that can run simultaneously.

So with a proper Makefile the problem could be solved.

.PHONY: all $(PHP_DEST)

# Create an array of targets in the form of PHP1 PHP2 ... PHP90
PHP_DEST := $(addprefix PHP, $(shell seq 1 1 90))

# Default target
all: $(PHP_DEST)

# Run the proper script for each target
$(PHP_DEST):
        N=$(subst PHP,,$@); php path$N/some_job_$N.php

It creates 90 of PHP# targets each calls php path#/some_job_#.php. If You run make -j 5 then it will run 5 instance of php parallel. If one finishes it starts the next.

I renamed the Makefile to parallel.mak, I run chmod 700 parallel.mak and I added #!/usr/bin/make -f to the first line. Now it can be called as ./parallel.mak -j 5.

Or even You can use the more sophisticated -l switch:

  -l [load], --load-average[=load]
        Specifies  that  no new jobs (commands) should be started if there
        are others jobs running and the load average is at least  load  (a
        floating-point number).  With no argument, removes a previous load
        limit.

In this case make will decide how many jobs can be launched depending on the system's load.

I tested it with ./parallel.mak -j -l 1.0 and run nicely. It started 4 programs in parallel at first contrary -j without args means run as many process parallel as it can.

这篇关于在任何给定时间运行 5 个脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！