GNU Parallel:如何确定作业的“位置"?您正在使用? [英] GNU Parallel: How do determine job "slot" you're using?

问题描述

我试图找到一种方法来确定parallel中当前正在使用的命令的插槽"或​​核心"作业.例如，我们都看到了parallel如何分发命令的类似图像:

I'm trying to find a way to determine the job "slot" or "core" a command is currently using in parallel. For example, we've all seen a similar image of how parallel distributes commands:

如果我想知道某个过程在哪一列中，我怎么知道?

If I want to know which column a certain process is in, how do I know?

我的具体问题得到了说明:如果将-j 4设置为一次只允许运行4个作业，我想动态知道一个命令占用的插槽是1 2 3或4.问题是我有一些命令无法可以并行运行，但是如果我知道我正在运行哪个插槽，那一切都很好.

My specific problem illustrated: if set -j 4 to only allow 4 jobs to run at once, I want to dynamically know which slot a command is taking, 1 2 3 or 4. The problem is I have some commands that cannot run in parallel, but if I knew which slot I was running in, I'm all good.

再举一个例子，说我正在并行执行以下命令:

Further example, say I have these commands I'm parallelizing:

command resource1 file1.rb
command resource2 file2.rb
command resource3 file3.rb
command resource4 file4.rb
command resource1 file5.rb
command resource2 file6.rb
command resource3 file7.rb
command resource4 file8.rb

一次只能使用一个命令来使用每个资源.假设我像往常一样一次将这些命令放在parallel中，一次执行4个作业，并且作业3完成并转到队列中的下一个，我现在并行运行这些指令:

Only one command can use each resource at a time. Say I put these commands in parallel like usual with 4 jobs at a time, and job 3 finishes up and it goes to the next in the queue, I now have these running in parallel:

command resource1 file1.rb
command resource2 file2.rb
command resource3 file3.rb
command resource1 file5.rb

注意resource1被两个命令使用，不好.我需要一个环境变量或告诉下一个命令使用资源号4的内容，以便并行化的命令如下所示:

Notice resource1 is being used by two commands, not good. What I need is an environment variable or something to tell the next command to use resource number 4, so that the parallelized commands looks like this:

command resource1 file1.rb
command resource2 file2.rb
command resource3 file3.rb
command resource4 file5.rb

我曾经考虑过使用文件系统或其他一些正在使用资源的外部标志，但是我认为使用并行进程时，可能会出现竞争条件.

I've thought about using the filesystem or some other kind of external signs of which resources are in use, but I figure with parallel processes, there are likely to be race conditions going that route.

我已经看了遍，任何帮助都将不胜感激！

I've looked all over, any help is greatly appreciated!

推荐答案

我相信您正在寻找{％}:

I believe you are looking for {%}:

parallel -j4 command ressource{%} file{}.rb ::: {1..8}

这篇关于GNU Parallel:如何确定作业的“位置"?您正在使用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！