安装带有cupy的pip软件包会使安装永无止境地循环 [英] Installing a pip package with cupy as a requirement puts install in never ending loop

问题描述

我试图制作一个带有cupy的pip包作为要求之一，但是我将cupy包含在要求中，pip安装最终以永无止境的循环结束.我正在尝试在已经安装了Cupy的Google Colab上安装该软件包，因此它应该只检查Cupy是否已经安装，而不要尝试再次安装.

I am trying to make a pip package with cupy as one of the requirements, but I include cupy in the requirement, the pip install ends up in a never ending loop. I am trying to install the package on Google Colab, which already has Cupy install, so it should only check if Cupy is already installed and not try to install it again.

我在github中做了一个最小的pip包，其中cupy是唯一的要求.

I made a minimal pip package in github where cupy is the only requirement.

https://github.com/Santosh-Gupta/TroubleShootCupyInstall

我尝试使用

!pip install --verbose https://github.com/Santosh-Gupta/TroubleShootCupyInstall/archive/master.zip --log 'file.log'

输出很多，因为它很冗长，但是这些是到达循环时要打印的行.

The output is a lot since it is in verbose, but these are the lines which are printed when it gets to its loop.

 x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_FORCE_INLINES=1 -I/usr/local/cuda/include -I/usr/include/python3.6m -c cupy/cuda/nvtx.cpp -o build/temp.linux-x86_64-3.6/cupy/cuda/nvtx.o
  x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/cupy/cuda/nvtx.o -L/usr/local/cuda/lib64 -lnvToolsExt -o build/lib.linux-x86_64-3.6/cupy/cuda/nvtx.cpython-36m-x86_64-linux-gnu.so -Wl,--disable-new-dtags,-rpath,/usr/local/cuda/lib64
  building 'cupy.cuda.thrust' extension
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_FORCE_INLINES=1 -I/usr/local/cuda/include -I/usr/include/python3.6m -c cupy/cuda/thrust.cpp -o build/temp.linux-x86_64-3.6/cupy/cuda/thrust.o
  NVCC options: ['--generate-code=arch=compute_30,code=compute_30', '--generate-code=arch=compute_50,code=compute_50', '--generate-code=arch=compute_60,code=sm_60', '--generate-code=arch=compute_61,code=sm_61', '--generate-code=arch=compute_70,code=sm_70', '--generate-code=arch=compute_75,code=sm_75', '--generate-code=arch=compute_70,code=compute_70', '-O2', '--compiler-options="-fPIC"']
  /usr/local/cuda/bin/nvcc -D_FORCE_INLINES=1 -I/usr/local/cuda/include -I/usr/include/python3.6m -c cupy/cuda/cupy_thrust.cu -o build/temp.linux-x86_64-3.6/cupy/cuda/cupy_thrust.o --generate-code=arch=compute_30,code=compute_30 --generate-code=arch=compute_50,code=compute_50 --generate-code=arch=compute_60,code=sm_60 --generate-code=arch=compute_61,code=sm_61 --generate-code=arch=compute_70,code=sm_70 --generate-code=arch=compute_75,code=sm_75 --generate-code=arch=compute_70,code=compute_70 -O2 --compiler-options="-fPIC"

为方便起见，我制作了一个Google Colab笔记本，它可以运行此行并具有完整的输出.

For convenience I made a Google Colab notebook which runs this line, and has the full output.

https://colab.research.google.com/drive/1DFR78cJ07KaHkJfph8370SxNw0HXI50

https://colab.research.google.com/drive/1DFR78cJ07KaHkJfpjh8370SxNw0HXI50

推荐答案

CuPy当前提供名为cupy的源包和名为cupy-cudaXX的二进制分发包(其中XX是CUDA版本). 当前，由于cupy-cuda100使用的是CUDA 10.0，因此它随附了Google Colab. 如果您指定cupy作为软件包的要求，即使已经通过cupy-cuda100提供了CuPy，也会下载并安装cupy源软件包(需要几分钟的构建).

CuPy currently provides source package named cupy and binary distribution packages named cupy-cudaXX (where XX is a CUDA version). Currently Google Colab is shipped with cupy-cuda100 because it is using CUDA 10.0. If you specify cupy as an requirement of your package, cupy source package will be downloaded and installed (requires build which takes several minutes), even though CuPy is already available through cupy-cuda100.

不幸的是，Python软件包分发工具(例如setuptools，pip等)没有提供良好处理这种复杂的软件包组成的方法.

Unfortunately, Python package distribution tools (such as setuptools, pip, etc.) do not provide a way to well-handle this kind of complex package composition.

在setup.py(或包装的__init__.py)中

try:
  import cupy
except Exception:
  raise RuntimeError('CuPy is not available. Please install it manually: https://docs-cupy.chainer.org/en/stable/install.html#install-cupy')

# You can also use `cupy.__version__` and `numpy.lib.NumpyVersion` to test CuPy version requirement here.

解决方法2

像在Chainer中一样，使用pkg_resources(属于setuptools的一部分)手动检查需求.

Workaround 2

Manually check requirements using pkg_resources (part of setuptools) as done in Chainer.

• https://github.com/chainer /chainer/blob/v6.3.0/chainer/_environment_check.py#L44
• https://github.com /chainer/chainer/blob/v6.3.0/chainer/_version.py#L4-L18

• https://github.com/chainer/chainer/blob/v6.3.0/chainer/_environment_check.py#L44
• https://github.com/chainer/chainer/blob/v6.3.0/chainer/_version.py#L4-L18

这篇关于安装带有cupy的pip软件包会使安装永无止境地循环的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！