如何减少anaconda目录中的文件数? [英] How to reduce the number of files in the anaconda directory?

问题描述

我正在计算集群上运行conda环境，其中每个项目"的文件总数受到限制(最多200k个文件).我只创建了几个conda环境(用于Python 2.7的anaconda；每个环境中安装的〜200 python& R软件包；环境之间的高软件包重叠)，并且已经达到该文件数限制.即使使用conda clean -a，也只会删除一小部分文件.我的conda环境中的某些python软件包(例如boost)包含的文件超过1万个，而clean并不能减少这种情况.

I'm running conda environments on a compute cluster where the total number of files per "project" is restricted (200k files max). I've only created a couple of conda environments (anaconda for Python 2.7; ~200 python & R packages installed in each environment; high package overlap between environments) and already hit that file number limit. Even when using conda clean -a only a small fraction of the files are removed. Some python packages in my conda environments (e.g., boost) contain >10k files, and clean does not reduce this.

有什么办法可以极大地减少作为conda环境一部分存储的文件数量?

Is there any way to greatly reduce the number of files stored as part of a conda environment?

推荐答案

Anaconda使用硬链接来减少占用的磁盘空间.但是，如果对文件数量施加限制，则每个硬链接都会计数.

Anaconda uses hard links to reduce the consumed disk space. But if a limit is imposed on the number of files, each hard link counts.

如评论中所述，使用Miniconda代替Anaconda并仅安装实际需要的软件包可能会有所帮助.

As discussed in the comments, using Miniconda instead of Anaconda, and installing only the packages you actually need, might help.

如果这还不够，我建议将您的多个环境合并为一个.这样，重叠的软件包的硬链接就会更少了.当然，这与环境的使用相反，但这就是解决方法的本质.

If this isn't enough, I'd recommend to merge several of your environments into one. Then you'll have fewer hardlinks for the packages that overlap. Of course that is the opposite of what environments are there for, but such is the nature of workarounds.

这篇关于如何减少anaconda目录中的文件数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！