如何在pytorch中为ppc64le架构安装torchmeta？ [英] How does one install torchmeta for a ppc64le architecture in pytorch?

查看：201
本文介绍了如何在pytorch中为ppc64le架构安装torchmeta？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我试图在ppc64le体系结构中使用 torchmeta 。不幸的是，由于ppc64le需要特殊的二进制文件才能进行安装，因此安装起来并不容易。

I was trying to use torchmeta in a ppc64le architecture. Unfortunately it's not been easy to install since ppc64le requires special binaries to work.

我最终通过遵循这些说明（在右边的ibm频道前面加上conda二进制文件，还要安装所有必需的文件）：

I eventually managed to get the right binaries for pytorch and torchvision by following these instructions (that prepend the right ibm channel with the conda binaries, plus installs all the required files too):

conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
conda create -n my_new_env python=3.7 powerai=1.7.0
conda activate my_new_env

我继续安装正确版本的torchmeta，它是 1.3.1 ，因为ppc64le仅具有pytorch 1.3.1 和Torrvision 0.4.2 。所以我做到了：

after that I proceeded to install the right version of torchmeta, which was 1.3.1 since ppc64le only has pytorch 1.3.1 and torchvision 0.4.2. So I did:

pip install torchmeta==1.3.1

，但是现在我遇到一个新错误，即它找不到与我想要执行的操作兼容的正确版本的h5py。错误消息很大，要粘贴，但我会粘贴我希望有用的部分：

but now I have a new error that it cannot find the right version of h5py compatible with what I want to do. The error message is to large to paste but I will paste what I hope are useful part of it:

(my_new_env) [miranda9@hal-login ~]$ pip install torchmeta==1.3.1
Collecting torchmeta==1.3.1
  Using cached torchmeta-1.3.1-py3-none-any.whl (144 kB)
Requirement already satisfied: requests in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (2.22.0)
Requirement already satisfied: torchvision<0.6.0,>=0.4.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (0.4.2)
Requirement already satisfied: torch<1.5.0,>=1.3.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.3.1)
Processing ./.cache/pip/wheels/87/f5/ad/9f04a48453875e8054c19f9fe3f50cbbe0c09b956835555019/Pillow-6.2.2-cp37-cp37m-linux_ppc64le.whl
Requirement already satisfied: numpy>=1.14.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.17.4)
Requirement already satisfied: tqdm>=4.0.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (4.36.1)
Collecting h5py~=2.9.0
  Using cached h5py-2.9.0.tar.gz (287 kB)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (2020.6.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (1.25.10)
Requirement already satisfied: idna<2.9,>=2.5 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (2.8)
Requirement already satisfied: six in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchvision<0.6.0,>=0.4.0->torchmeta==1.3.1) (1.13.0)
Building wheels for collected packages: h5py
  Building wheel for h5py (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ccg1oj0n
       cwd: /tmp/pip-install-bpmeop26/h5py/
  Complete output (1321 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-ppc64le-3.7
  creating build/lib.linux-ppc64le-3.7/h5py
  copying h5py/__init__.py -> build/lib.linux-ppc64le-3.7/h5py
  copying h5py/h5py_warnings.py -> build/lib.linux-ppc64le-3.7/h5py
  copying h5py/highlevel.py -> build/lib.linux-ppc64le-3.7/h5py
  copying h5py/ipy_completer.py -> build/lib.linux-ppc64le-3.7/h5py
  copying h5py/version.py -> build/lib.linux-ppc64le-3.7/h5py
  creating build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/attrs.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/base.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/compat.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/dataset.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/datatype.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/dims.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/files.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/filters.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/group.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/selections.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/selections2.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/vds.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
  creating build/lib.linux-ppc64le-3.7/h5py/tests
  copying h5py/tests/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests
  copying h5py/tests/common.py -> build/lib.linux-ppc64le-3.7/h5py/tests
  creating build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_attrs.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_attrs_data.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_base.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_dataset.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_datatype.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_dimension_scales.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_file.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_file_image.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_group.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5d_direct_chunk_write.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5f.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5p.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5t.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_objects.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_selections.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_slicing.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
  creating build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_attribute_create.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dataset_getitem.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dataset_swmr.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_datatype.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_deprecation.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dims_dimensionproxy.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_file.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_filters.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_threads.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
  creating build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_highlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_virtual_source.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  running build_ext
  Autodetected HDF5 1.10.2
  ********************************************************************************
                         Summary of the h5py configuration
  
      Path to HDF5: None
      HDF5 Version: '1.10.2'
       MPI Enabled: False
  Rebuild Required: True
  
  ********************************************************************************
  Executing api_gen rebuild of defs
  Executing cythonize()
  [ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
  /tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd
    tree = Parsing.p_module(s, pxd, full_module_name)

...

  /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   #warning "Using deprecated NumPy API, disable it with " \
    ^
  In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
  /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
   #include "hdf5.h"
                    ^
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for h5py
  Running setup.py clean for h5py
Failed to build h5py
DEPRECATION: Could not build wheels for h5py which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: Pillow, h5py, torchmeta
  Attempting uninstall: Pillow
    Found existing installation: Pillow 7.1.2
    Uninstalling Pillow-7.1.2:
      Successfully uninstalled Pillow-7.1.2
  Attempting uninstall: h5py
    Found existing installation: h5py 2.8.0
    Uninstalling h5py-2.8.0:
      Successfully uninstalled h5py-2.8.0
    Running setup.py install for h5py ... error
    ERROR: Command errored out with exit status 1:
     command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py


... 

    copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
    copying h5py/tests/hl/test_vds/test_virtual_source.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
    running build_ext
    Autodetected HDF5 1.10.2
    ********************************************************************************
                           Summary of the h5py configuration
    
        Path to HDF5: None
        HDF5 Version: '1.10.2'
         MPI Enabled: False
    Rebuild Required: True
    
    ********************************************************************************
    Executing cythonize()
    [ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
    /tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd


...

    warning: h5py/api_types_hdf5.pxd:730:6: 'H5Z_ERROR_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:731:6: 'H5Z_DISABLE_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:732:6: 'H5Z_ENABLE_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:733:6: 'H5Z_NO_EDC' redeclared
    building 'h5py.defs' extension
    creating build/temp.linux-ppc64le-3.7
    creating build/temp.linux-ppc64le-3.7/tmp
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py
    gcc -pthread -B /home/miranda9/.conda/envs/my_new_env/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DH5_USE_16_API -I./h5py -I/tmp/pip-install-bpmeop26/h5py/lzf -I/opt/local/include -I/usr/local/include -I/home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include -I/home/miranda9/.conda/envs/my_new_env/include/python3.7m -c /tmp/pip-install-bpmeop26/h5py/h5py/defs.c -o build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py/defs.o
    In file included from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1830:0,
                     from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                     from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:26,
                     from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:
    /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it with " \
      ^
    In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
    /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
     #include "hdf5.h"
                      ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of h5py
  Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py
   from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py
  Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py-2.8.0-py3.7.egg-info
   from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py-2.8.0-py3.7.egg-info
ERROR: Command errored out with exit status 1: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py Check the logs for full command output.

任何人都知道如何在ppc64le中成功安装有效的torchmeta版本（使用wmcle 1.7.0）

anyone know how I can successfully install a working torchmeta version in a ppc64le (using wmcle 1.7.0)?

相关：

IBM gitissue支持torchmeta： https://github.com/IBM/powerai/issues/269

IBM gitissue for torchmeta support: https://github.com/IBM/powerai/issues/269

h5py gitissue for torchmeta： https://github.com/h5py/h5py/issues/1678

h5py gitissue for torchmeta: https://github.com/h5py/h5py/issues/1678

IBM h5py对torchmeta的支持： https://github.com/IBM/powerai/issues/270

IBM h5py support for torchmeta: https://github.com/IBM/powerai/issues/270

推荐答案

因为在那儿不是用于h5py的powerpc的轮子，而是从源代码（从tarball）安装h5py。这要求同时提供Python和h5py开发标头，请参见 https： //docs.h5py.org/en/stable/build.html#source-installation 。

Because there are not wheels for powerpc for h5py you are installing h5py from source (from the tarball). This requires both the Python and h5py development headers to be available, see https://docs.h5py.org/en/stable/build.html#source-installation.

从conda安装h5py或安装所需的构建依赖项。

Either install h5py from conda or install the required build dependencies.

这篇关于如何在pytorch中为ppc64le架构安装torchmeta？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
Python最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆