在任何给定时间运行 5 个脚本 [英] Have 5 scripts running at any given time
问题描述
我有一个启动 90 个不同 PHP 脚本的 bash 脚本(在 CentOS 6.4 下运行),即.
#!/bin/bashphp path1/some_job_1.php&php path2/some_job_2.php&php path3/some_job_3.php&php path4/some_job_4.php&php path5/some_job_5.phpphp path6/some_job_6.php&php path7/some_job_7.php&php path8/some_job_8.php&php path9/some_job_9.php&php path10/some_job_10.php...出口 0
为了避免我的服务器超载,我使用 & 符号 &
,它可以工作,但 我的目标是始终让 5 个脚本同时运行
有没有办法做到这一点?
这个问题被弹出了好几次,但我找不到合适的答案.我想现在我找到了一个很好的解决方案!
不幸的是 parallel
不是标准发行版的一部分,而是 make 是.它有一个开关 -j
来做并行.
man make(1)]: (更多关于 make 并行执行的信息)
<块引用> -j [jobs], --jobs[=jobs]指定同时运行的作业(命令)的数量.如果-j 选项不止一个,最后一个有效.如果-j 选项不带参数给出,make 不会限制可以同时运行的作业数.
因此,使用适当的 Makefile
可以解决问题.
.PHONY: all $(PHP_DEST)# 创建一个目标数组,形式为 PHP1 PHP2 ... PHP90PHP_DEST := $(添加前缀 PHP, $(shell seq 1 1 90))# 默认目标全部:$(PHP_DEST)# 为每个目标运行正确的脚本$(PHP_DEST):N=$(subst PHP,,$@);php 路径$N/some_job_$N.php
它会创建 90 个 PHP#
目标,每个调用 php path#/some_job_#.php
.如果您运行 make -j 5
那么它将运行 5 个 php 实例并行.如果一个完成,它会开始下一个.
我将 Makefile
重命名为 parallel.mak
,我运行 chmod 700 parallel.mak
并添加了 #!/usr/bin/make -f
到第一行.现在可以调用 ./parallel.mak -j 5
.
甚至你可以使用更复杂的 -l
开关:
-l [load], --load-average[=load]指定不应该启动新的作业(命令),如果有其他作业是否正在运行并且平均负载至少为负载(a浮点数).不带参数,删除先前的负载限制.
在这种情况下,make 将决定有多少可以根据系统负载启动作业.
我用 ./parallel.mak -j -l 1.0
对其进行了测试并且运行良好.它首先并行启动了 4 个程序
I have a bash script (running under CentOS 6.4) that launches 90 different PHP scripts, ie.
#!/bin/bash
php path1/some_job_1.php&
php path2/some_job_2.php&
php path3/some_job_3.php&
php path4/some_job_4.php&
php path5/some_job_5.php
php path6/some_job_6.php&
php path7/some_job_7.php&
php path8/some_job_8.php&
php path9/some_job_9.php&
php path10/some_job_10.php
...
exit 0
In order to avoid overloading my server, I use the ampersand &
, it works, but my goal is to always have 5 scripts running at the same time
Is there a way to achieve this?
This question is popped several times, but I could not find a proper answer for it. I think now I found a good solution!
Unfortunately parallel
is not the part of the standard distributions, but make is. It has a switch -j
to do makes parallel.
man make(1)]: (more info on make's parallel execution)
-j [jobs], --jobs[=jobs] Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.
So with a proper Makefile
the problem could be solved.
.PHONY: all $(PHP_DEST)
# Create an array of targets in the form of PHP1 PHP2 ... PHP90
PHP_DEST := $(addprefix PHP, $(shell seq 1 1 90))
# Default target
all: $(PHP_DEST)
# Run the proper script for each target
$(PHP_DEST):
N=$(subst PHP,,$@); php path$N/some_job_$N.php
It creates 90 of PHP#
targets each calls php path#/some_job_#.php
. If You run make -j 5
then it will run 5 instance of php parallel. If one finishes it starts the next.
I renamed the Makefile
to parallel.mak
, I run chmod 700 parallel.mak
and I added #!/usr/bin/make -f
to the first line. Now it can be called as ./parallel.mak -j 5
.
Or even You can use the more sophisticated -l
switch:
-l [load], --load-average[=load] Specifies that no new jobs (commands) should be started if there are others jobs running and the load average is at least load (a floating-point number). With no argument, removes a previous load limit.
In this case make will decide how many jobs can be launched depending on the system's load.
I tested it with ./parallel.mak -j -l 1.0
and run nicely. It started 4 programs in parallel at first contrary -j
without args means run as many process parallel as it can.
这篇关于在任何给定时间运行 5 个脚本的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!