点构建选项以使用多核 [英] Pip build option to use multicore

问题描述

我发现pip编译软件包时仅使用单核.由于某些python软件包使用pip构建需要花费一些时间，因此我想在计算机上利用多核.使用Makefile时，我可以执行以下命令:

I found that pip only use single core when it compiles packages. Since some python packages takes some time to build using pip, I'd like to utilize multicore on the machine. When using Makefile, I can do that like following command:

make -j4

如何为点子实现相同的目的?

How can I achieve same thing for pip?

推荐答案

据我所知，pip似乎不具备这种能力，但我可能会误会.

From what I can tell it does not look like pip has this ability but I may be mistaken.

要在python中执行多重处理，请使用多重处理程序包，[这是我找到的指南]( http://pymotw.com/2/multiprocessing/basics.html )，如果您有兴趣的话，该方法是 Multiprocessing vs Threading Python ，以确保多处理可以完成我的工作以为这样做了，因为它利用了多个CPU.

To do multiprocessing in python you use the multiprocessing package, [here is a guide I found] (http://pymotw.com/2/multiprocessing/basics.html) about how to do it if you are interested and this is a link to the python docs that talk about it. I also found this question useful, Multiprocessing vs Threading Python, to make sure that multiprocessing did what I thought it did, being take advantage of multiple CPUs.

我已经遍历了pip源代码(在此处)可以找到对多处理程序包的引用并且未发现该包的任何用途.这意味着pip不使用/不支持多处理.据我所知，/pip/commands/install.py文件是您所关心的问题之一，因为在您运行pip install <package>时便会调用它.对于此文件，具体导入为

I have gone through the pip source code (available here) looking for a reference to the multiprocessing package and did not find any use of the package. This would mean that pip does not use/support multiprocessing. From what I can tell the /pip/commands/install.py file is the one of interest for your question as it is called when you run pip install <package>. For this file specifically the imports are

from __future__ import absolute_import

import logging
import os
import tempfile
import shutil
import warnings

from pip.req import InstallRequirement, RequirementSet, parse_requirements
from pip.locations import virtualenv_no_global, distutils_scheme
from pip.basecommand import Command
from pip.index import PackageFinder
from pip.exceptions import (
    InstallationError, CommandError, PreviousBuildDirError,
)
from pip import cmdoptions
from pip.utils.deprecation import RemovedInPip7Warning, RemovedInPip8Warning

您可以看到它没有对多处理程序包的任何引用，但是为了确定我确实检查了所有其他文件.

which you can see does not have any reference to the multiprocessing package but I did check all of the other files just to be sure.

此外，我查看了 pip安装文档，并找到了没有参考使用多个内核进行安装.

Furthermore, I checked the pip install documentation and found no reference to installing using multiple cores.

TL; DR:Pip不能满足您的要求.我可能没看错，因为我没看很久，但是我很确定它不支持.

TL;DR: Pip doesn't do what you are asking. I may be wrong as I didn't look at the source that long but I'm pretty sure it just doesn't support it.

这篇关于点构建选项以使用多核的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！