MPI任务计划 [英] MPI Task Scheduling

问题描述

我想使用MPI开发一个任务调度程序，其中有一个主处理器，并且有工作/客户端处理器.每个工作人员都拥有需要计算的所有数据，但可以从主节点获取要处理的索引.计算后，工作人员将一些数据返回给主服务器.问题在于某些过程会很快，而某些过程会很慢. 如果我运行一个循环，以便主机在每次迭代中发送和接收(阻塞/非阻塞)数据，则它无法继续进行下一步，直到它从分配给它的上一个索引接收到当前工作者的数据为止.底线是，如果一个工作人员花费太多时间进行计算，那么它将成为限制因素，即使使用非阻塞技术，主节点也无法继续为下一个工作人员分配索引.是否可以跳过分配给工人的工作，然后继续下一个工作.

I want to develop a task scheduler using MPI where there is a single master processor and there are worker/client processors. Each worker has all the data it needs to compute, but gets the index to work on from the master. After the computation the worker returns some data to the master. The problem is that some processes will be fast and some will be slow. If I run a loop so that at each iteration the master sends and receives (blocking/non-blocking) data then it can't proceed to next step till it has received data from the current worker from the previous index assigned to it. The bottom line is if a worker takes too long to compute then it becomes the limiting factor and the master can't move on to assign an index to the next worker even if non-blocking techniques are used. Is it possible to skip assigning to a worker and move on to next.

我开始认为MPI可能不是这样做的范例. python是执行任务调度的好平台吗?

I'm beginning to think that MPI might not be the paradigm to do this. Would python be a nice platform to do task scheduling?

推荐答案

使用MPI_Irecv()和MPI_Test()绝对可能.主进程需要做的就是为每个工作进程发布一个无阻塞的接收，然后在一个循环测试中为输入的数据进行每个测试.如果处理完成，则向其发送新索引，为其发布新的非阻塞接收，然后继续.

This is absolutely possible using MPI_Irecv() and MPI_Test(). All the master process needs to do is post a non-blocking receive for each worker process, then in a loop test each one for incoming data. If a process is done, send it a new index, post a new non-blocking receive for it, and continue.

这篇关于MPI任务计划的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！