使用gnu-parallel无限期地执行分叉过程，该过程会捕获各个退出错误并重新生成 [英] Fork processes indefinetly using gnu-parallel which catch individual exit errors and respawn
问题描述
我想标题给了你这个想法.
I guess the title gives you this thought.
另一个重复的问题
Another duplicate question
好吧，让我详细解释一下.
Well, let me explain this in detail.
好的，我们走了.
我正在使用Gearman来处理一堆任务.我有一个齿轮工客户，可以将此任务发送给工人.要同时运行这些任务，必须有更多的工人同时处理一个任务.目前，我根据cpus数量创建工人.就我而言，其为4.因此， 4个进程.
I am using gearman to handle stack of tasks. I have a gearman client which send this task to workers. To run these task concurrently, there must be more workers to handle a task at a time. Presently, I create workers as per number of cpus. In my case, its 4. So, 4 processes.
./worker & ./worker & ./worker & ./worker.
我有同时运行的同一文件.但是，我没有它们各自的PID和他们的退出代码状态. 我要他们永远跑.另外，此过程不会在控制台cuz上输出任何内容，因为它们会传达 客户端-工作人员风格 .最大的问题是保持终端运行.记住，我希望这个进程永远运行.
I have same file running concurrently. But, I don't have their respective PIDs & their exit code status. I want them to run forever. Also, this processes do not output anything on console cuz they communicate client - worker style. And the biggest problem is to keep the terminal running. Remember, I want this processes running forever.
现在，为了解决此问题，我决定创建一个 Upstart服务，该服务在后台运行此过程.但是，我想确保我所有的工人都在运转.然后我遇到了 gnu-parallel ，它似乎是一个完美的工具.我找不到完美的命令.而且，我没有时间探索所有内容.
Now, to solve this problem, I decided to create a Upstart service which run this processes in background. But, I want to make sure that all my workers are running. Then I came across gnu-parallel which seems to be a perfect tool. I can't find the perfect command. And, I don't have time to explore it all.
所以，我想做以下事情.
So, I want to do the followings.
- 在新贵公司中使用gnu-parallel执行并发工作者.我有的
现在是这段代码.
seq 8 | parallel -n0 ./worker - 如果这些工人中的任何一个崩溃并退出，且代码> 0，我想 使用退出代码记录pid并重新启动工作进程.
- use gnu-parallel in upstart to exec concurrent workers. what I have
now is this code.
seq 8 | parallel -n0 ./worker - If any of these workers crashes and exits with code > 0, I want to log the pid with exit code and restart the worker process.
这是我的暴发户服务
# workon
description "worker load"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
script
cpu="$(nproc)"
line="./worker"
for i in `seq 2 ${cpu}`; do
line="${line} & ./worker"
done
sh -c "echo $$ > test.log; ${line}"
end script
我需要上述代码中的 parallel 实现.
I need parallel implementation in above code.
上面代码中的缺陷是，如果最后一个工作程序被杀死，它将使用所有4工作程序进程重新生成服务.例如.
The flaw in the above code is that it re-spawns the service with all 4 worker process if the last worker get killed. For example.
___________________
Name | PID
worker 1011
worker 1012
worker 1013
worker 1014
如果PID 1014被杀死，则服务重新生成更多的4工人+老3工人.总共是7.
If the PID 1014 get killed than the service respawn more 4 workers + old 3 workers. Which comes to 7 in total.
如何使用 gnu-parallel 使所有4名工作人员在后台服务中存活?
How to use gnu-parallel to keep all 4 workers alive in background service?
谢谢.
推荐答案
GNU Parallel具有--joblog，在这里可能会有所帮助:
GNU Parallel has --joblog that may be helpful here:
seq 1000000000000 | parallel -N0 --joblog out.log worker
这将使每个CPU内核启动一个工作线程.当工作人员崩溃时，将记录退出代码.但是，PID不会.
This will start one worker per CPU core. When a worker crashes, the exitcode will be logged. The PID, however, will not.
该工作线程将不会重新启动，但是将启动一个新的工作线程，因此每个CPU内核始终将有一个工作线程.当1000000000000工人崩溃时，GNU Parallel将不会启动另一个.如果您认为它太小，则增加1000000000000(在31700年中每秒增加1，对于大多数人来说就足够了，但是如果您是Vulcan，情况可能会有所不同.)
The worker will not be restarted, but a new worker will be started so there will always be one per CPU core running. When 1000000000000 workers have crashed, then GNU Parallel will not start another. Increase 1000000000000 if you think it is too small (it is 1 for each second in 31700 years - it will be enough for most humans, but if you are Vulcan, things may be different).
如果您真的需要pid，则可以执行以下操作:
If you really need the pid, you can probably do something like:
seq 1000000000000 | parallel -N0 --joblog out.log 'echo $$; exec worker' >pids
如果您只需要GNU并行的PID，则:
If you only need the PID of GNU Parallel:
seq 1000000000000 | parallel -N0 --joblog out.log worker &
echo $!
这篇关于使用gnu-parallel无限期地执行分叉过程，该过程会捕获各个退出错误并重新生成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
