conda环境可以继承基本软件包吗? [英] Can conda environment inherit base packages?
问题描述
我正在寻找环境 do 从根继承的解决方案，但是寻找答案似乎有很多困惑.许多OP问题认为，它们不是在继承程序包.因此，搜索结果找到了这些问题，但是答案却有相反的解决方案(或者只是解释说错了).
I'm looking for a solution where environments do inherit from root, but searching for the answer there seems to be a lot of confusion. Many OP questions believe they are inheriting packages when they are not. So, the search results find these questions, but the answer has the counter solution (or just explain they are mistaken).
也就是说，一个OP实际上具有相似的目标.可以在Anaconda环境之间共享软件包吗?该OP表示它们正在运行硬盘空间不足.暗示共享"应该在新环境中使用相同的已安装软件包.答案(不接受)是使用-clone .
That said, one OP actually has a similar objective. Can packages be shared across Anaconda environments? This OP says they are running out of space on their HDD. The implication being "sharing" should use the same installed packages in the new environment. The answer (not accepted) is to use --clone.
我还发现了这篇帖子，新创建的conda env是否从基本env继承所有软件包?，其中说-clone 不共享软件包.在这篇文章中，OP认为他们的新环境是共享"程序包，然后得出结论共享"程序包不存在.非分隔水蟒环境的用途是什么?
I also found this post, Do newly created conda envs inherit all packages from the base env? which says --clone does not share packages. In this post the OP believed their new environment "shared" packages, and then concludes "shared" packages don't exist.
What is the use of non-separated anaconda environments?
我测试了-clone 标志和选项.两个环境目录具有相同的大小:2G +.
I tested both the --clone flag, and the Conda Docs instructions to "build identical environments" options. Both env directories have the same identical size: 2G+.
(base) $ conda list --explicit > spec-file.txt
# Produced Size On Disk: 2.14 GB (2,305,961,984 bytes)
(base) conda create --name myclone --clone root
# Produced Size On Disk, clone: 2.14 GB (2,304,331,776 bytes)
唯一的区别是在相同的环境中构建版本，再次下载了软件包，然后克隆复制了本地文件，花费的时间更少了..
The only difference was building identical environment downloaded the packages again, and clone coppied the local file taking much less time.
我使用Miniconda将CLI工具部署到同事工作站.基本上，当我需要添加基本安装中不需要的特定模块时，所有工具都使用相同的软件包，但偶尔会例外.
I use Miniconda to deploy CLI tools to coworker workstations. Basically, the tools all use the same packages, with the occasional exception, when I need to add a particular module which I don't want in the base install.
目标是在扩展基本软件包(类似于 virtualenv --system-site-packages )的环境中使用conda create，而不要复制它们的环境安装.
The goal is to use conda create for environments that extend the base packages similar to virtualenv --system-site-packages, and not to duplicate their installation.
Responding to @merv and his link to this post (Why are packages installed rather than just linked to a specific environment?) which says Conda venvs inherit base packages by default. I had another opportunity this weekend with the problem. Here is the base case:
下载了 Miniconda安装程序.已安装设置
Downloaded the Miniconda installer. Installed with settings
- 为我安装
- 安装位置:(C:\ Users \ xtian \ Miniconda3_64)注意:我添加了
_64 - 高级选项
- 将Anaconda添加到系统PATH变量False
- 将Anaconda注册为系统Python 3.7，True
我更新了pip和设置工具，
conda更新pip设置工具
conda update pip setuptools
下面，我在
base中列出软件包:Below, I list packages in
base:(base) C:\Users\xtian>conda list # packages in environment at C:\Users\xtian\Miniconda3_64: # # Name Version Build Channel asn1crypto 1.3.0 py37_0 ca-certificates 2020.1.1 0 certifi 2019.11.28 py37_0 cffi 1.13.2 py37h7a1dbc1_0 chardet 3.0.4 py37_1003 conda 4.8.2 py37_0 conda-package-handling 1.6.0 py37h62dcd97_0 console_shortcut 0.1.1 3 cryptography 2.8 py37h7a1dbc1_0 idna 2.8 py37_0 menuinst 1.4.16 py37he774522_0 openssl 1.1.1d he774522_3 pip 20.0.2 py37_1 powershell_shortcut 0.0.1 2 pycosat 0.6.3 py37he774522_0 pycparser 2.19 py37_0 pyopenssl 19.1.0 py37_0 pysocks 1.7.1 py37_0 python 3.7.4 h5263a28_0 pywin32 227 py37he774522_1 requests 2.22.0 py37_1 ruamel_yaml 0.15.87 py37he774522_0 setuptools 45.1.0 py37_0 six 1.14.0 py37_0 sqlite 3.31.1 he774522_0 tqdm 4.42.0 py_0 urllib3 1.25.8 py37_0 vc 14.1 h0510ff6_4 vs2015_runtime 14.16.27012 hf0eaf9b_1 wheel 0.34.2 py37_0 win_inet_pton 1.1.0 py37_0 wincertstore 0.2 py37_0 yaml 0.1.7 hc54c509_2然后我成功创建了新的venv:
(base) C:\Users\xtian>conda create -n wsgiserver Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: C:\Users\xtian\Miniconda3_64\envs\wsgiserver Proceed ([y]/n)? y Preparing transaction: done Verifying transaction: done Executing transaction: done在这里，我激活新的wsgiserver虚拟环境，列出软件包，最后使用
pip进行测试-但没有pip！我今天用64位和32位安装程序进行了测试:Here I activate the new wsgiserver virtual environment, list packages, and finally test with
pip--but there is nopip! I tested today with 64 and 32 bit installers:(base) C:\Users\xtian>conda activate wsgiserver (wsgiserver) C:\Users\xtian>conda list # packages in environment at C:\Users\xtian\Miniconda3_64\envs\wsgiserver: # # Name Version Build Channel (wsgiserver) C:\Users\xtian>pip 'pip' is not recognized as an internal or external command, operable program or batch file.推荐答案
Conda环境是否应该继承基本程序包?
否.建议的工作流程是使用
conda create --clone创建一个新的独立环境，然后对该环境进行更改以添加其他软件包.或者，可以将模板环境转储到YAML(conda env export> env.yaml )，对其进行编辑以包括或删除程序包，然后从中创建一个新环境( condaenv创建-f env.yaml -n foo ). Should Conda environments inherit base packages?
No. The recommended workflow is to use
conda create --cloneto create a new standalone environment, and then mutate that environment to add additional packages. Alternatively, one can dump the template environment to a YAML (conda env export > env.yaml), edit it to include or remove packages, and then create a new environment from that (conda env create -f env.yaml -n foo).在大多数情况下，对这种浪费存储的担忧是没有根据的. 1 由于Conda的使用，可能会有许多新环境比实际占用的空间更大.硬链接以最大程度地减少冗余.在问题
Concern about this wasting storage is unfounded in most situations.1 There can be a mirage of new environments taking up more space than they really do, due to Conda's use of hardlinks to minimize redundancy. A more detailed analysis of this can be found in the question, Why are packages installed rather than just linked to a specific environment?.
不支持，但是有可能.首先，让我们明确声明此GitHub Issue 中可以找到对此的更详细的讨论.
It's not supported, but it's possible. First, let's explicitly state that nested activation of Conda environments via the
conda activate --stackcommand does not enable or help to allow inheritance of Python packages across environments. This is because it does not manipulatePYTHONPATH, but instead only keeps the previous active environment onPATHand skips the deactivate scripts. A more detailed discussion of this is available in this GitHub Issue.现在，我们避免了红色鲱鱼，让我们谈谈
PYTHONPATH.可以使用此环境变量来包含其他要搜索的site-packages目录.所以，天真地像Now that we've avoided that red herring, let's talk about
PYTHONPATH. One can use this environment variable to include additionalsite-packagesdirectories to search. So, naively, something likeconda activate foo PYTHONPATH=$CONDA_ROOT/lib/python3.7/site-packages python应使用可用的 base 和 foo 软件包启动Python.这项工作的主要限制是，新环境中的Python必须与 base 的版本匹配，直到次要版本(在本例中为3.7.*)为止.
should launch Python with the packages of both base and foo available to it. A key constraint for this to work is that the Python in the new environment must match that of base up to and including the minor version (in this case 3.7.*).
虽然这将实现程序包继承，但我们需要考虑:这实际上会节省空间吗?我认为实际上可能不会，而且这里为什么.
While this will achieve package inheritance, we need to consider: Will this actually conserve space? I'd argue that in practice it likely won't, and here's why.
大概我们不想在物理上复制Python安装，但是新环境必须安装Python，以帮助限制对所需新软件包的求解.为此，我们不仅应该匹配Python版本(
conda create -n foo python = 3.7)，而且应该与 base 完全相同的构建:Presumably, we don't want to physically duplicate the Python installation, but the new environment must have a Python installed in order to help constrain solving for the new packages we want. To do this, we should not only match the Python version (
conda create -n foo python=3.7), but rather the exact same build as base:# first check base's python conda list -n base '^python$' # EXAMPLE RESULT # Name Version Build Channel python 3.7.6 h359304d_2 # use this when creating the environment conda create -n foo python=3.7.6=h359304d_2这将使Conda进行链接，并在两种环境中使用相同的物理副本.但是，不能保证Python的依赖项也会重用 base 中的软件包.实际上，如果有任何兼容的较新版本可用，它将下载并安装这些版本.
This will let Conda do its linking thing and use the same physical copy in both environments. However, there is no guarantee that Python's dependencies will also reuse the packages in base. In fact, if any compatible newer versions are available, it will download and install those.
此外，假设我们现在安装
scikit-learn:Furthermore, let's say that we now install
scikit-learn:conda install -n foo scikit-learn这将再次检查它的最新版本及其依赖项，而与这些依赖关系的较旧但兼容的版本是否已经可以通过 base 获得.因此，不必要将更多软件包安装到软件包缓存中.
This again is going to check for the newest versions of it and its dependencies, independent of whether older but compatible versions of those dependencies are already available through base. So, more packages are unnecessarily being installed into the package cache.
这里的模式似乎是我们真的想找到一种方法来让 foo env安装新软件包，但要使用尽可能多的现有软件包来满足依赖性.而这正是
conda create --clone已经完成的工作. 2The pattern here seems to be that we really want to find a way to have the foo env install new packages, but use as many of the existing packages to satisfy dependencies. And that is exactly what
conda create --clonealready does.2因此，我完全失去了继承的动力.
Hence, I lose the motivation to bother with inheritance altogether.
我推测，对于纯Python包的特殊情况，使用
pip install --target从 base 环境中安装，以将与 base 兼容的软件包安装到外部基本.然后，用户可以从 base 启动python之前，将该目录添加到PYTHONPATH中.I'd speculate that for the special case of pure Python packages it may be plausible to use
pip install --targetfrom the base environment to install packages compatible with base to a location outside of base. The user could then add this directory toPYTHONPATHbefore launchingpythonfrom base.这不是我的首选.我知道克隆策略是可以管理的；我不知道从长期来看会发生什么.
This would not be my first choice. I know the clone strategy is manageable; I wouldn't know what to expect with this going long-term.
[1] 这将一直存在，只要程序包缓存的位置(
pkgs_dirs)以及创建环境的位置(默认为envs_dirs)在同一卷上.具有多个卷的配置应该使用软链接，这最终会产生相同的效果.除非有人手动禁用这两种类型的链接，否则Conda会在静默减少冗余方面做得不错.[1] This will hold as long as the locations of the package cache (
pkgs_dirs) and where the environment is created (which defaults toenvs_dirs) are on the same volume. Configurations with multiple volumes should be using softlinks, which will ultimately have the same effect. Unless one has manually disabled linking of both types, Conda will do a decent job at silently minimizing redundancy.[2] 从技术上讲，有人可能会企图使用
-offline标志来强制Conda使用已经缓存的内容.但是，OP的前提是附加软件包是新软件包，因此假设我们在缓存中已经具有兼容版本可能不是明智的选择.[2] Technically, one might also have a stab at using the
--offlineflag to force Conda to use what it already has cached. However, the premise of OP is that the additional package is new, so it may not be wise to assume we already have a compatible version in the cache.这篇关于conda环境可以继承基本软件包吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
