列表不会随着Ray parallel python改变 [英] Lists won't change with Ray parallel python

问题描述

我的问题是，如果我重新分配列表中的项目，以便在并行过程中发生重新分配，那么在并行过程完成后，更改将恢复为原始状态.

My issue is that if I reassign an item in a list such that the reassignment happens during a parallel process, then after the parallel processes are finished, the change reverts back to its original state.

在下面的示例中(为了易于理解而大大简化了)，我有一个将列表元素NoZeros [0]更改为鸡肉"的函数，以及另一个将NoZeros [1]更改为三明治"的函数.我什至在第二个函数中添加了"global"，只是为了证明这不是局部问题还是全局问题，它看起来像一个问题，但实际上并非如此.正如打印命令在运行程序时向您显示的那样，列表元素确实发生了变化.问题在于，在这些过程之后调用NoZeros时，NoZeros就是从此开始的，而不是"[" chicken，" sandwich].

In the below example- greatly simplified for ease of understanding-, I have a function that changes the list element NoZeros[0] to "chicken" and a second function that changes NoZeros[1] to "sandwich". I even put "global" in the second function just to demonstrate that this isn't a local vs global issue- it looks like one, but it indeed is not. As the print commands show you when you run the program, the list elements do actually change. The issue is that when calling NoZeros after these processes, NoZeros is what it was to begin with instead of "["chicken", "sandwich"].

我知道python的多处理程序包具有完全相同的问题，但是它通过将您不希望恢复的内容一步一步"解决并在其之前拍打"manager.list()"来解决..我的问题是我无法终生弄清楚Ray的意义.例如，在Python的多处理库中，您只需在NoZeros更改之前在某个地方编写NoZeros = manager.list(NoZeros)，这将是结尾，但是我找不到Ray的等效项或是否存在甚至是等效的.

I know that python's multiprocessing package has exactly the same issue, but it was solved by taking "one step up" of whatever it is you wanted to not revert back to and slapping "manager.list()" at the before it. My issue is that I can't for the life of me figure out what the equivalent is for Ray. For example, in Python's multiprocessing library, you would just write NoZeros=manager.list(NoZeros) somewhere before NoZeros gets altered, and that would be the end of it, but I can't find what equivalent there is for Ray or if there even is an equivalent.

如何使用RAY并行更改列表?非常感谢.

HOW DO I CHANGE LISTS IN PARALLEL USING RAY? Many thanks.

也:请注意，此脚本可能使您陷入循环，因为您可能最终在并行进程完成之前打印NoZeros.这是我遇到的另一个错误，请多加注意，但这不是优先事项.我要说明的一点是，您可能想在下一个单元格中运行print(NoZeros)(至少Jupyter具有此功能).在python的多处理库中，只需执行"process.join()"即可解决问题所在的进程，因此使我想到了额外的问题:

Also: note that this script may throw you for a loop, because you may end up printing NoZeros BEFORE the parallel processes finish. This is another bug I'm having trouble with and would appreciate attention on, but it is not the priority. The point I'm trying to make is that you probably want to run print(NoZeros) in the next cell (Jupyter has this functionality, at least). In python's multiprocessing library, one would just do "process.join()" and that would solve end the process(es) in question, so that brings me to the bonus question:

奖金问题:如何使ray.wait()工作?我如何告诉我的代码仅在前一个命令(即使它们是并行命令)完成后才继续执行下一个命令?

Bonus question: How do i get ray.wait() to work; how do I tell my code to only proceed to my next command if the previous commands- even if these are parallel commands- are finished?

'''

import ray

ray.shutdown()
ray.init()

NoZeros=[0,0]

@ray.remote
def Chicken():
    print("NoZeros[0] is",NoZeros[0],"but will change to chicken")
    NoZeros[0]="chicken"
    print("Now, NoZeros[0] is",NoZeros[0])


@ray.remote
def GlobalSandwich():
    global NoZeros #This is just to show that "global" doesn't solve anything
    print("NoZeros[1] is",NoZeros[1],"but will change to sandwich")
    NoZeros[1]="sandwich"
    print("Now, NoZeros[1] is",NoZeros[1])

Chicken.remote()
GlobalSandwich.remote()

#Unhash these 3 lines of code if youd like to try tackling another question: why does ray.wait() not work? 
#How do i wait until parallel processes end, so i can continue my code?

#WaitList=[Chicken.remote(),GlobalSandwich.remote()]
#ray.wait(WaitList)
#print("If you see this first, ray.wait() isnt working")


#This line of code right here was executed in the next command line (Jupyter); this print command happens when the other processes are finished  
print(NoZeros)

'''

推荐答案

使用Ray，可变的全局状态应位于演员.例如，您可以执行以下操作:

With Ray, mutable global state should live in Actors. For example, you could do something like:

@ray.remote
class ListActor:
    def __init__(self, l):
        self._list = l

    def get(self, i):
        return self._list[i]

    def set(self, i, val):
        self._list[i] = val

    def to_list(self):
        return self._list

然后，为了使用它，您可以将其作为参数传递(同样，您不应该依赖于全局变量).

Then in order to use it, you can pass it in as a parameter (again, you shouldn't rely on global variables).

NoZeros = ListActor.remote([0,0])

@ray.remote
def Chicken(NoZeros):
    print("NoZeros[0] is",ray.get(NoZeros.get.remote(0)),"but will change to chicken")
    NoZeros.set(0, "chicken")
    print("Now, NoZeros[0] is",ray.get(NoZeros.get(0)))


# We need to make sure this function finishes executing before we print.
ray.get(Chicken.remote(NoZeros))

print(ray.get(NoZeros.to_list.remote()))

这篇关于列表不会随着Ray parallel python改变的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！