从python代码连接到远程python内核 [英] Connect to remote python kernel from python code

问题描述

我一直在使用 PaperMill 定期执行我的python笔记本.要执行计算密集型笔记本，我需要连接到在我的EMR集群中运行的远程内核.

I have been using PaperMill for executing my python notebook periodically. To execute compute intensive notebook, I need to connect to remote kernel running in my EMR cluster.

对于Jupyter笔记本，我可以通过使用jupyter notebook --gateway-url=http://my-gateway-server:8888启动jupyter服务器来做到这一点，并且能够在远程内核上执行我的代码.但是，如何让我的本地python代码(通过PaperMill)使用远程内核?在内核管理器中进行什么更改以连接到远程内核?

In case of Jupyter notebook I can do that by starting jupyter server with jupyter notebook --gateway-url=http://my-gateway-server:8888 and I am able to execute my code on remote kernel. But how do I let my local python code(through PaperMill) to use remote kernel? What changes do what to make in Kernel Manager to connect to remote kernel?

我可以找到的一个相关的SO答案是此处.这建议将端口转发到远程服务器，并使用服务器中的连接文件初始化KernelManager.我无法执行此操作，因为Ipython.zmp中不再包含blockingkernelmanager，并且我也希望像jupyter那样使用HTTP连接.

One related SO answer I could find is here. This suggests to do port forwarding to remote server and initialize KernelManager with the connection file from the server. I am not able to do this as blockingkernelmanager is no longer in Ipython.zmp and I would also prefer HTTP connection like how jupyter does.

推荐答案

我已经用简单的步骤编写了这个Word文档.希望您会发现它有用.我的方法使用Jupyter笔记本.您可以ugnore风筝部分.您所要做的就是在编辑器中使用氢气.这是

I have written this word document with concise steps to the setup. I hope you will find it useful. My method uses Jupyter notebook. You can ugnore the kite part. All you have to do is use hydrogen with your editor. Here it is

https://drive.google.com/file/d /1INVxvJVrnoj8z0iBqesYa1F1lbHgFSGu/view?usp = drivesdk

这篇关于从python代码连接到远程python内核的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！