当内存不足时，Jupyter Lab会冻结计算机-如何防止它发生? [英] Jupyter Lab freezes the computer when out of RAM - how to prevent it?

问题描述

我最近开始使用Jupyter Lab，我的问题是我使用的是非常大的数据集(通常，数据集本身大约是计算机RAM的1/4).经过几次转换后，另存为新的Python对象，我倾向于耗尽内存.问题是，当我接近可用的RAM限制并执行需要其他RAM空间的任何操作时，计算机将冻结，而修复它的唯一方法是重新启动它.这是Jupyter Lab/Notebook中的默认行为，还是我应该设置的某些设置?通常，我希望程序崩溃(例如在RStudio中)，而不是整个计算机

I have recently started using Jupyter Lab and my problem is that I work with quite large datasets (usually the dataset itself is approx. 1/4 of my computer RAM). After few transformations, saved as new Python objects, I tend to run out of memory. The issue is that when I'm approaching available RAM limit and perform any operation that needs another RAM space my computer freezes and the only way to fix it is to restart it. Is this a default behaviour in Jupyter Lab/Notebook or is it some settings I should set? Normally, I would expect the program to crash out (as in RStudio for example), not the whole computer

推荐答案

绝对最可靠的解决方案是使用Docker容器.您可以指定要分配给Jupyter的内存量，并且如果容器用完了内存，那也没什么大不了的(只记得经常保存即可，但这不用多说).

Absolutely the most robust solution to this problem would be to use Docker containers. You can specify how much memory to allocate to Jupyter, and if the container runs out of memory it's simply not a big deal (just remember to save frequently, but that goes without saying).

此博客将为您提供大部分帮助那里的路.也有一些不错的说明，可以从此处免费提供的，官方维护的Jupyter映像之一设置Jupyter Lab:

This blog will get you most of the way there. There are also some decent instructions setting up Jupyter Lab from one of the freely available, officially maintained, Jupyter images here:

https://medium.com/fundbox-engineering/overview-d3759e83969c

，然后您可以按照本教程中的说明将docker run命令修改为(例如3GB):

and then you can modify the docker run command as described in the tutorial as (e.g. for 3GB):

docker run --memory 3g <other docker run args from tutorial here>

有关docker内存选项的语法，请参见以下问题:

For syntax on the docker memory options, see this question:

docker哪个单元运行"--memory" ;选项期望吗?

这篇关于当内存不足时，Jupyter Lab会冻结计算机-如何防止它发生?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！