Poetry 如何处理二进制依赖项?(尤其是麻木) [英] How does Poetry work regarding binary dependencies? (esp. numpy)

问题描述

到目前为止，我一直使用 conda 作为虚拟环境和依赖项管理.但是，将我的 environment.yml 文件从我的开发机器传输到生产服务器时，有些东西无法按预期工作.现在，我想研究替代方案.诗歌看起来不错，尤其是因为

Until now I have been using conda as virtual environment and dependency management. However, some stuff does not work as expected when transfering my environment.yml file from my development machine to the production server. Now, I would like to look into alternatives. Poetry seems nice, especially because

poetry 还维护一个锁文件，它比 pipenv 有好处，因为它跟踪哪些包是子依赖项.(https://realpython.com/effective-python-environment/#poetry)

poetry also maintains a lock file, and it has a benefit over pipenv because it keeps track of which packages are subdependencies. (https://realpython.com/effective-python-environment/#poetry)

这可能会大大提高稳定性.但是，我正在从事科学重项目(矩阵、数据科学、机器学习)，所以实际上，我需要 scipy 堆栈(例如 numpy、pandas、scitkit-learn).

which might improve stability quite a bit. However, I am working on science-heavy projects (matrices, data science, machine learning), so in practise I need the scipy stack (e.g. numpy, pandas, scitkit-learn).

Python 对于一些纯计算工作负载来说变得太慢了，所以 numpy 和 scipy 诞生了.[...] 它们是用 C 语言编写的，只是包装成一个 python 库.

Python became too slow for some pure computational workloads so numpy and scipy were born. [...] They are written in C and just wrapped as a python library.

编译此类库会带来一系列挑战，因为它们(或多或少)必须在您的机器上进行编译才能获得最大性能并与 glibc 等库正确链接.

Compiling such libraries brings a set of challenges since they (more or less) have to be compiled on your machine for maximum performance and proper linking with libraries like glibc.

Conda 是作为一种多合一的解决方案推出的，用于为科学界管理 Python 环境.

Conda was introduced as an all-in-one solution to manage python environments for the scientific community.

[...] 不是在您的机器上使用脆弱的库编译过程，而是预编译库，并在您请求它们时下载.不幸的是，该解决方案附带了一个警告 - conda 不使用 PyPI，这是最流行的 Python 包索引.

[...] Instead of using a fragile process of compiling libraries on your machine, libraries are precompiled and just downloaded when you request them. Unfortunately, the solution comes with a caveat - conda does not use PyPI, the most popular index of python packages.

(https://modelpredict.com/python-dependency-management-tools#fnref:conda-compiling-challenges)

据我所知，这甚至不符合 Conda 的标准，因为它做了相当多的优化以充分利用我的 CPU/GPU/numpy 架构.(https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/#Myth-#6:-现在-that-pip-uses-wheels,-conda-is-no-longer-necessary)

As far as I know, this doesn't even do Conda justice, because it does quite a bit of optimization to get the most out of my CPU/GPU/architecture for numpy. (https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/#Myth-#6:-Now-that-pip-uses-wheels,-conda-is-no-longer-necessary)

https://numpy.org/install/ 本身建议使用 conda，但也说可以通过pip安装(诗歌使用pypi)

https://numpy.org/install/ itself advises to use conda, but also says that one can install via pip (and poetry uses pypi)

对于根据个人喜好或阅读下文有关 conda 和 pip 之间的主要区别而了解的用户，他们更喜欢基于 pip/PyPI 的解决方案，我们建议:

For users who know, from personal preference or reading about the main differences between conda and pip below, they prefer a pip/PyPI-based solution, we recommend:

[...] 使用 Poetry 作为维护最完善的工具，以与 conda 类似的方式提供依赖项解析器和环境管理功能.

[...] Use Poetry as the most well-maintained tool that provides a dependency resolver and environment management capabilities in a similar fashion as conda does.

我想获得诗歌设置的稳定性和conda设置的速度.

I would like to get the stability of the poetry setup and the speed of the conda setup.

诗歌如何处理二进制依赖?它是否也像 conda 一样考虑我的硬件?

如果诗歌在这方面没有体现，我可以将它与 conda 结合起来吗?

If poetry not deliver in this regard, can I combine it with conda?

推荐答案

numpy 为不同的 os、cpu 架构和 python 版本提供了几个 wheel 文件.wheel 包是预编译的，所以目标系统不需要编译包.

numpy provides several wheel files for different os, cpu architecture and python versions. wheel packages are precompiled, so the target system doesn't have to compile the package.

poetry 可以根据您的系统为您选择合适的轮子.

poetry is able to choose the right wheel for you, depending on your system.

说到这里，我推荐使用poetry，只要你只需要python包，pypi也有.一旦您需要其他非 Python 工具，请坚持使用 conda.(免责声明:我是诗歌的维护者之一).

Saying this, I would recommend using poetry, as long as you just need python packages, which are also available at pypi. As soon as you need other, non-python tools, stick to conda. (Disclaimer: I'm one of the maintainer of poetry).

还相关:https://github.com/python-poetry/poetry/问题/1904

这篇关于Poetry 如何处理二进制依赖项?(尤其是麻木)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！