带有RPYC的多处理Python"ValueError:禁用酸洗" [英] Multiprocessing Python with RPYC "ValueError: pickling is disabled"
问题描述
我试图在rpyc
服务中使用多处理程序包,但是当我尝试从客户端调用公开函数时得到ValueError: pickling is disabled
.我知道multiprocesing
包使用酸洗在进程之间传递信息,并且rpyc
不允许酸洗,因为它是不安全的协议.因此,我不确定将多处理与rpyc一起使用的最佳方法(或者是否存在).如何在rpyc服务中使用多重处理?这是服务器端代码:
I am trying to use the multiprocessing package within an rpyc
service, but get ValueError: pickling is disabled
when I try to call the exposed function from the client. I understand that the multiprocesing
package uses pickling to pass information between processes and that pickling is not allowed in rpyc
because it is an insecure protocol. So I am unsure what the best way (or if there is anyway) to use multiprocessing with rpyc. How can I make use of multiprocessing within a rpyc service? Here is the server side code:
import rpyc
from multiprocessing import Pool
class MyService(rpyc.Service):
def exposed_RemotePool(self, function, arglist):
pool = Pool(processes = 8)
result = pool.map(function, arglist)
pool.close()
return result
if __name__ == "__main__":
from rpyc.utils.server import ThreadedServer
t = ThreadedServer(MyService, port = 18861)
t.start()
这是产生错误的客户端代码:
And here is the client side code that produces the error:
import rpyc
def square(x):
return x*x
c = rpyc.connect("localhost", 18861)
result = c.root.exposed_RemotePool(square, [1,2,3,4])
print(result)
推荐答案
您可以在协议配置中启用酸洗.配置存储为字典,您可以修改 default ,并将其传递到服务器(protocol_config
=)和客户端(config =
).您还需要定义要在客户端和服务器端并行化的功能.因此,这是server.py
的完整代码:
You can enable pickling in the protocol configuration. The configuration is stored as a dictionary, you can modify the default and pass it to both the server (protocol_config
= ) and client (config =
). You also need to define the function being parallelized on both the client and server side. So here is the full code for server.py
:
import rpyc
from multiprocessing import Pool
rpyc.core.protocol.DEFAULT_CONFIG['allow_pickle'] = True
def square(x):
return x*x
class MyService(rpyc.Service):
def exposed_RemotePool(self, function, arglist):
pool = Pool(processes = 8)
result = pool.map(function, arglist)
pool.close()
return result
if __name__ == "__main__":
from rpyc.utils.server import ThreadedServer
t = ThreadedServer(MyService, port = 18861, protocol_config = rpyc.core.protocol.DEFAULT_CONFIG)
t.start()
对于client.py
,代码为:
import rpyc
rpyc.core.protocol.DEFAULT_CONFIG['allow_pickle'] = True
def square(x):
return x*x
c = rpyc.connect("localhost", port = 18861, config = rpyc.core.protocol.DEFAULT_CONFIG)
result = c.root.exposed_RemotePool(square, [1,2,3,4])
print(result)
这篇关于带有RPYC的多处理Python"ValueError:禁用酸洗"的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!