朱莉娅@parallel for循环与return语句 [英] Julia @parallel for loop with return statement

问题描述

如何在满足条件后立即为所有工作人员返回的函数中编写并行for循环?

How can I write a parallel for loop in a function that returns for all workers as soon as a condition is met?

即像这样的东西:

function test(n)
  @sync @parallel for i in 1:1000
    {... statement ...}
    if {condition}
      return test(n+1)
    end
  end
end

所有工作人员都停止在for循环上工作，而只有主进程返回? (然后其他进程又开始与下一个for循环一起工作吗?)

where all the workers stop working on the for loop and only the main process returns? (and the other processes start again working with the next for loop?)

推荐答案

该问题似乎是执行令人尴尬的并行"搜索任务的基本模式. @parallel for构造非常适合分区工作，但没有break短路逻辑可作为for停止在单个处理流程中.

The question seems like a basic pattern for doing "embarrassingly parallel" search tasks. The @parallel for construct is good for partitioning work, but doesn't have the break short-circuit logic for stopping as the for in single process flow.

为了在Julia中演示如何执行此操作，请考虑一个玩具问题，该问题是找到带有多个轮子的密码锁的密码.可以使用某种方法(花费combodelay时间-参见下面的代码)检查车轮的每个设置是否正确.找到正确的车轮编号后，将搜索下一个车轮.高级伪代码类似于OP问题中给出的代码段.

To demonstrate how to do this in Julia consider a toy problem of finding the combination of a combination lock with several wheels. Each setting of a wheel can be checked for correctness with some method (taking a combodelay time - see code below). After the correct number for a wheel is found, the next wheel is searched. The high level pseudo code is like the snippet given in the OP question.

下面是运行代码(分别在0.5和0.6上)以执行此操作.一些注释解释了细节，并且代码以单个块的形式提供，以便于剪切和粘贴.

The following is running code (on 0.5 and 0.6) to do this. Some comments explain details, and the code is given in a single chunk for easy cut-and-paste.

# combination lock problem parameters
const wheel_max = 1000  # size of wheel
@everywhere const magic_number = [55,10,993]  # secret combination
const wheel_count = length(magic_number)  # number of wheels
const combodelay = 0.01 # delay time to check single combination

# parallel short-circuit parameters
const check_to_work_ratio = 160  # ratio to limit short-circuit overhead

function find_combo(wheel,combo=Int[])
  done = SharedArray{Int}(1)       # shared variable to hold if and what combo
  done[1] = 0                      #  succeded. 0 means not found yet
  # setup counters to limit parallel overhead
  @sync begin
    @everywhere global localdone = false
    @everywhere global checktime = 0.0
    @everywhere global worktime = 0.0
  end
  # do the parallel work
  @sync @parallel for i in 1:wheel_max
    global localdone
    global checktime
    global worktime
    # if not checking too much, look at shared variable
    if !localdone && check_to_work_ratio*checktime < worktime
      tic()
      localdone = done[1]>0
      checktime += toq()
    end
    # if no process found combo, check another combo
    if !localdone
      tic()
      sleep(combodelay) # simulated work delay, {..statement..} from OP
      if i==magic_number[wheel]    # {condition} from OP
        done[1] = i              
        localdone = true
      end
      worktime += toq()
    else
      break
    end
  end
  if done[1]>0 # check if shared variable indicates combo for wheel found
    push!(combo,done[1])
    return wheel<wheel_count ? find_combo(wheel+1,combo) : (combo,true)
  else
    return (combo,false)
  end
end

function find_combo_noparallel(wheel,combo=Int[])
  found = false
  i = 0
  for i in 1:wheel_max
    sleep(combodelay)
    if i==magic_number[wheel]
      found = true
      break
    end
  end
  if found
    push!(combo,i)
    return wheel<wheel_count ? 
      find_combo_noparallel(wheel+1,combo) : (combo,true)
  else
    return (combo,false)
  end
end

function find_combo_nostop(wheel,combo=Int[])
  done = SharedArray{Int}(1)
  done[1] = 0
  @sync @parallel for i in 1:wheel_max
    sleep(combodelay)
    if i==magic_number[wheel]
      done[1] = i
    end
  end
  if done[1]>0
    push!(combo,done[1])
    return wheel<wheel_count ? 
      find_combo_nostop(wheel+1,combo) : (combo,true)
  else
    return (combo,false)
  end
end

result = find_combo(1)
println("parallel with short-circuit stopping:       $result")
@assert result == (magic_number, true)

result = find_combo_noparallel(1)
println("single process with short-circuit stopping: $result")
@assert result == (magic_number, true)

result = find_combo_nostop(1)
println("parallel without short-circuit stopping:    $result")
@assert result == (magic_number, true)

println("\ntimings")

print("parallel with short-circuit stopping        ")
@time find_combo(1);
print("single process with short-circuit stopping  ")
@time find_combo_noparallel(1)
print("parallel without short-circuit stopping     ")
@time find_combo_nostop(1)

nothing

可能会有更好看的实现，并且一些元编程可以隐藏一些短路机制.但这应该是一个好的开始.

There could be better looking implementations, and some meta-programming can hide some of the short-circuit machinery. But this should be good as a start.

结果应大致如下:

parallel with short-circuit stopping:       ([55,10,993],true)
single process with short-circuit stopping: ([55,10,993],true)
parallel without short-circuit stopping:    ([55,10,993],true)

timings
parallel with short-circuit stopping          4.473687 seconds
single process with short-circuit stopping   11.963329 seconds
parallel without short-circuit stopping      11.316780 seconds

这是为演示3个工作进程而计算的.真正的问题应该有更多的过程和每个过程更多的工作，然后短路的好处将显而易见.

This is calculated for demonstration with 3 worker processes. Real problems should have many more processes and more work per process and then the benefits of short-circuiting will be evident.

这篇关于朱莉娅@parallel for循环与return语句的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！