在 Python 轮子中包含运行时依赖项 [英] Include run-time dependencies in Python wheels

问题描述

我想分发一个完整的 virtualenv，或者一堆具有运行时依赖项的精确版本的 Python 轮子，例如:

I'd like to distribute a whole virtualenv, or a bunch of Python wheels of exact versions with their runtime dependencies, for example:

• pycurl
  • pycurl.so
    • libcurl.so
      • libz.so
      • libssl.so
      • libcrypto.so
      • libgssapi_krb5.so
        • libkrb5.so
          • libresolv.so

          我想我可以依靠系统来安装 libssl.so，但肯定不是正确版本的 libcurl.so，也可能不是 Kerberos.

          I suppose I could rely on the system to have libssl.so installed, but surely not libcurl.so of the correct version and probably not Kerberos.

          将一个库与所有运行时依赖项打包到一个轮子中的最简单方法是什么?

          What is the easiest way to package one library in a wheel with all the run-time dependency?

          或者这是一个傻瓜的差事，我应该打包整个virtualenv?如何可靠地做到这一点?

          Or is that a fool's errand and I should package entire virtualenv? How to do that reliably?

          附言动态编译不是一种选择，某些模块已打补丁.

          P.S. compiling on the fly is not an option, some modules are patched.

          推荐答案

          AFAIK，没有很好的标准方法来可移植地安装包的依赖项.Continuum 有正是为此目的制作了conda.numpy 家伙在他们的包中编写了自己的 distutils 子模块来安装一些复杂的依赖项，现在 至少其中一些人提倡将 conda 作为解决方案.不幸的是，您可能需要自己为其中一些依赖项制作 conda 包.

          AFAIK, there is no good standard way to portably install dependencies with your package. Continuum has made conda for precisely this purpose. The numpy guys wrote their own distutils submodule in their package to install some complicated dependencies, and now at least some of them advocate conda as a solution. Unfortunately, you may have to make conda packages for some of these dependencies yourself.

          如果你没有可移植性，那么针对目标机器的包管理器显然会起作用.否则，对于便携式包管理器，conda 是我所知道的唯一选择.

          If you're fine without portability, then targeting the package manager of the target machines will obviously work. Otherwise, for a portable package manager, conda is the only option I know of.

          或者，从您的帖子(即时编译不是一种选择")听起来便携性对您来说可能不是问题，在这种情况下，您还可以安装所有要求以前缀目录(我遇到的大多数安装程序都支持 configure --prefix=/some/dir/ 选项).如果您有一个有保证的单一架构，您可能可以将所有依赖项前缀安装到一个目录中，然后像文件一样传递它.conda 方法可能更简洁，但我已经使用了很多前缀安装，而且它们往往是最简单的解决方案之一.

          Alternatively, from your post ("compiling on the fly is not an option") it sounds like portability may not be an issue for you, in which case you could also install all the requirements to a prefix directory (most installers I've come across support a configure --prefix=/some/dir/ option). If you have a guaranteed single architecture, you could probably prefix-install all your dependencies to a single directory and pass that around like a file. The conda approach would probably be cleaner, but I've used prefix installs quite a bit and they tend to be one of the easiest solutions to get going.

          至于 conda，它同时是一个包管理器和一个类似virtualenv"的环境/python 安装.虽然 virtualenv 添加在现有 python 安装之上，但 conda 接管了整个安装，因此您可以更确定所有依赖项都已考虑在内.与 pip 相比，它旨在添加通用的非 Python 依赖项，而不仅仅是编译 C/Cpp 扩展.有关更多信息，我会看到:

          As for conda, it is simultaneously a package-manager and a "virtualenv"-like environment/python install. While virtualenv is added on top of an existing python install, conda takes over the whole install, so you can be more sure that all the dependencies are accounted for. Compared to pip, it is designed for adding generalized non-Python dependencies, instead of just compiling C/Cpp exentions. For more info, I would see:

          • pip vs conda(也推荐 buildout 作为一种可能性)
          • conda 作为 python 安装
          • pip vs conda (also recommends buildout as a possibility)
          • conda as a python install

          至于如何将 conda 用于您的目的，文档 解释了如何创建食谱:

          As for how to use conda for your purpose, the docs explain how to create a recipe:

          构建一个包需要一个配方.配方是平面目录，其中包含以下文件:

          Conda build framework

          Building a package requires a recipe. A recipe is flat directory which contains the following files:

          • meta.yaml(元数据文件)
          • build.sh(使用 bash 执行的 Unix 构建脚本)
          • bld.bat(使用 cmd 执行的 Windows 构建脚本)
          • run_test.py(可选的 Python 测试文件)
          • 对源的补丁(可选，见下文)
          • 其他资源，不包含在源中，不能被由构建脚本生成.
          • meta.yaml (metadata file)
          • build.sh (Unix build script which is executed using bash)
          • bld.bat (Windows build script which is executed using cmd)
          • run_test.py (optional Python test file)
          • patches to the source (optional, see below)
          • other resources, which are not included in the source and cannot be generated by the build scripts.

          应该使用相同的配方在所有平台上构建包.

          The same recipe should be used to build a package on all platforms.

          在构建包时，会调用以下步骤:

          When building a package, the following steps are invoked:

          1. 读取元数据
          2. 下载源代码(到缓存中)
          3. 在源目录
          中提取源
          4. 应用补丁
          5. 创建构建环境(此处安装构建依赖项)
          6. 运行实际的构建脚本.当前工作目录是源设置了环境变量的目录.构建脚本安装到构建环境
          7. 执行一些必要的后期处理步骤:shebang、rpath 等.
          8. 将 conda 元数据添加到构建环境
          9. 将构建环境中的新文件打包成conda包
          10. 测试新的 conda 包:
              • 使用包(及其依赖项)创建测试环境
              • 运行测试脚本
          1. read the metadata
          2. download the source (into a cache)
          3. extract the source in a source directory
          4. apply the patches
          5. create a build environment (build dependencies are installed here)
          6. run the actual build script. The current working directory is the source directory with environment variables set. The build script installs into the build environment
          7. do some necessary post processing steps: shebang, rpath, etc.
          8. add conda metadata to the build environment
          9. package up the new files in the build environment into a conda package
          10. test the new conda package:
              • create a test environment with the package (and its dependencies)
              • run the test scripts

          在 conda-recipes 中有许多 conda 包的示例配方<https://github.com/continuumio/conda-recipes>_repo.

          :ref:conda skeleton <skeleton_ref> 命令可以帮助制作常见的骨架配方存储库，例如 PyPI _.

          The :ref:conda skeleton <skeleton_ref> command can help to make skeleton recipes for common repositories, such as PyPI <https://pypi.python.org/pypi>_.

          然后，作为客户端，您将安装包 类似于安装方式来自点子

          Then, as a client, you would install the package similar to how you would install from pip

          最后，docker 可能对你也很有趣，虽然我还没有看到它被广泛用于Python.

          Lastly, docker may also be interesting to you, though I haven't seen it much used for Python.

          这篇关于在 Python 轮子中包含运行时依赖项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！