Pip安装被杀死-内存不足-如何解决? [英] Pip install killed - out of memory - how to get around it?
本文介绍了Pip安装被杀死-内存不足-如何解决?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试安装spaCy，这是用于自然语言处理的库.
root@vps:~# python3 -m pip install spacy
Collecting spacy
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: spacy
Killed
一段时间后，它被杀死了.我已经在Stack Overflow上寻找答案并尝试过，这是由于内存不足(我的VPS上有1,2 Gb的可用空间，无法扩展)所致.快速tail -f /var/log/kern.log给我Out of memory: Kill process 31729 (python3) score 558 or sacrifice child.
我已经尝试添加--no-cache-dir，但这没有帮助.手动复制文件时似乎缺少某些内容.有什么方法可以在低RAM服务器上安装此软件包?
python3 -m pip install spacy -vvv
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Collecting spacy
1 location(s) to search for versions of spacy:
* https://pypi.python.org/simple/spacy/
Getting page https://pypi.python.org/simple/spacy/
Looking up "https://pypi.python.org/simple/spacy/" in the cache
Returning cached "301 Moved Permanently" response (ignoring date and etag info rmation)
Looking up "https://pypi.org/simple/spacy/" in the cache
Current age based on date: 29458
Freshness lifetime from max-age: 600
Freshness lifetime from request max-age: 600
Starting new HTTPS connection (1): pypi.org
https://pypi.org:443 "GET /simple/spacy/ HTTP/1.1" 304 0
Analyzing links from page https://pypi.org/simple/spacy/
<!-- Found link, skipped linked (hundreds of lines) -->
Found link https://files.pythonhosted.org/packages/a7/84/f35f44560f0ab9ef23c 4a334a513f2b6c8a95d7b3347ef68a0247da63c08/spacy-2.1.6.tar.gz#sha256=b708bb582597 6220de78ad7e20e827912ca95653738a812cc2e5d194e3060ae8 (from https://pypi.org/simp le/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*), version: 2.1 .6
Using version 2.1.6 (newest of versions: 0.31, 0.32, 0.33, 0.40, 0.51, 0.52, 0 .60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.67, 0.68, 0.70, 0.80, 0.81, 0.82, 0.83, 0.8 4, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.97, 0.98, 0.99, 0.100.0, 0.100.1, 0.100.2, 0.100.3, 0.100.4, 0.100.5, 0.100.6, 0.100.7, 0 .101.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.2.0, 1.3.0, 1. 4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.5, 1.8.0, 1.8.1, 1.8.2 , 1.9.0, 1.10.0, 1.10.1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.0.12, 2.0.13, 2.0.15, 2.0.16, 2.0.17, 2.0.18, 2 .1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6)
Looking up "https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247 babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_6 4.whl" in the cache
Current age based on date: 98439
Freshness lifetime from max-age: 365000000
The response is "fresh", returning cached response
365000000 > 98439
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd2938424 7babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_ 64.whl
Downloading from URL https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3b d29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylin ux1_x86_64.whl#sha256=2441ad9e6195669c13f4035dd77c4206414a40ebb8543c69f3386d984a bfb942 (from https://pypi.org/simple/spacy/) (requires-python:>=2.7,!=3.0.*,!=3. 1.*,!=3.2.*,!=3.3.*)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3 .5/dist-packages (from spacy)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/pytho n3.5/dist-packages (from spacy)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /usr/local/lib/python3.5/ dist-packages (from spacy)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /usr/local/lib/python3.5 /dist-packages (from spacy)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.5/dist-pa ckages (from spacy)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/l ocal/lib/python3.5/dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-p ackages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/di st-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5 /dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.5/d ist-packages (from thinc<7.1.0,>=7.0.8->spacy)
Installing collected packages: spacy
Killed
解决方案
已经在github上报道了这一点
https://github.com/explosion/spaCy/issues/3396
下面是一些有关如何解决此问题的好线索
https://chirale.org/2017/01/15/memory-error-on-pip-install-solved/
摘要中，以下是两个选项:
-
尝试使用
--no-cache-dir，很遗憾，这对您没有帮助 -
增加交换空间
$ create swap file of 512 MB
dd if=/dev/zero of=/swapfile bs=1024 count=524288
$ modify permissions
chown root:root /swapfile
chmod 0600 /swapfile
$ setup swap area
mkswap /swapfile
$ turn swap on
swapon /swapfile
I'm trying to install spaCy, a library for natural language processing.
root@vps:~# python3 -m pip install spacy
Collecting spacy
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: spacy
Killed
After a while, it's killed. I've already looked for answers on Stack Overflow and tried, and it's due to low memory (1,2 Gb free on my VPS, can't extend). A quick tail -f /var/log/kern.log gives me Out of memory: Kill process 31729 (python3) score 558 or sacrifice child.
I've already tried adding --no-cache-dir, but it didn't help. Something seems missing when copying the files manually. Is there any way to install this package on a low-RAM server?
EDIT : Complete log for python3 -m pip install spacy -vvv
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Collecting spacy
1 location(s) to search for versions of spacy:
* https://pypi.python.org/simple/spacy/
Getting page https://pypi.python.org/simple/spacy/
Looking up "https://pypi.python.org/simple/spacy/" in the cache
Returning cached "301 Moved Permanently" response (ignoring date and etag info rmation)
Looking up "https://pypi.org/simple/spacy/" in the cache
Current age based on date: 29458
Freshness lifetime from max-age: 600
Freshness lifetime from request max-age: 600
Starting new HTTPS connection (1): pypi.org
https://pypi.org:443 "GET /simple/spacy/ HTTP/1.1" 304 0
Analyzing links from page https://pypi.org/simple/spacy/
<!-- Found link, skipped linked (hundreds of lines) -->
Found link https://files.pythonhosted.org/packages/a7/84/f35f44560f0ab9ef23c 4a334a513f2b6c8a95d7b3347ef68a0247da63c08/spacy-2.1.6.tar.gz#sha256=b708bb582597 6220de78ad7e20e827912ca95653738a812cc2e5d194e3060ae8 (from https://pypi.org/simp le/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*), version: 2.1 .6
Using version 2.1.6 (newest of versions: 0.31, 0.32, 0.33, 0.40, 0.51, 0.52, 0 .60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.67, 0.68, 0.70, 0.80, 0.81, 0.82, 0.83, 0.8 4, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.97, 0.98, 0.99, 0.100.0, 0.100.1, 0.100.2, 0.100.3, 0.100.4, 0.100.5, 0.100.6, 0.100.7, 0 .101.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.2.0, 1.3.0, 1. 4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.5, 1.8.0, 1.8.1, 1.8.2 , 1.9.0, 1.10.0, 1.10.1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.0.12, 2.0.13, 2.0.15, 2.0.16, 2.0.17, 2.0.18, 2 .1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6)
Looking up "https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247 babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_6 4.whl" in the cache
Current age based on date: 98439
Freshness lifetime from max-age: 365000000
The response is "fresh", returning cached response
365000000 > 98439
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd2938424 7babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_ 64.whl
Downloading from URL https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3b d29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylin ux1_x86_64.whl#sha256=2441ad9e6195669c13f4035dd77c4206414a40ebb8543c69f3386d984a bfb942 (from https://pypi.org/simple/spacy/) (requires-python:>=2.7,!=3.0.*,!=3. 1.*,!=3.2.*,!=3.3.*)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3 .5/dist-packages (from spacy)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/pytho n3.5/dist-packages (from spacy)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /usr/local/lib/python3.5/ dist-packages (from spacy)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /usr/local/lib/python3.5 /dist-packages (from spacy)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.5/dist-pa ckages (from spacy)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/l ocal/lib/python3.5/dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-p ackages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/di st-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5 /dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.5/d ist-packages (from thinc<7.1.0,>=7.0.8->spacy)
Installing collected packages: spacy
Killed
解决方案
This has been reported already on github
https://github.com/explosion/spaCy/issues/3396
Below are some good thread on how to solve the issue
fork() failing with Out of memory error
Memory error while using pip install Matplotlib
out of memory issue in installing packages on Ubuntu server
https://chirale.org/2017/01/15/memory-error-on-pip-install-solved/
In Summary, below are the two options:
Try with
--no-cache-dir, which unfortunately has not helped youIncrease the swap space
$ create swap file of 512 MB
dd if=/dev/zero of=/swapfile bs=1024 count=524288
$ modify permissions
chown root:root /swapfile
chmod 0600 /swapfile
$ setup swap area
mkswap /swapfile
$ turn swap on
swapon /swapfile
这篇关于Pip安装被杀死-内存不足-如何解决?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
