Caffe多CPU构建 [英] Caffe multi CPU build

问题描述

我正在尝试在VirtualBox中的Ubuntu 14.04 x64上使用CPU_ONLY模式下的openblas构建 Caffe . 环境安装脚本，

I'm trying to build Caffe on Ubuntu 14.04 x64 in VirtualBox with openblas in CPU_ONLY mode.(Enviroment install script , Makefile.config )

我也不编译 OpenBlas ，而是通过apt-get像sudo apt-get -y install libopenblas-dev一样安装它，可能是问题的原因吗?

Also I'm not compiling OpenBlas, but install it via apt-get like sudo apt-get -y install libopenblas-dev, can it be reason of the problem?

设置了这些变量中的任何一个之后，速度都没有提高，并且在htop中我只看到一个CPU使用率.

After I set any of this variables, there is no speed improvement and in htop I see only one CPU utilisation.

export OPENBLAS_NUM_THREADS=4
export GOTO_NUM_THREADS=4
export OMP_NUM_THREADS=4

如何检查Caffe是否使用多个线程/CPU?

How to check if Caffe use several threads / CPUs?

更新:

我在 MNIST示例上尝试了caffe二进制文件，它占用了400％的CPU

I tried caffe binary on MNIST example and it utilise 400% of CPU.

1个线程

I0520 15:58:09.749832 12424 caffe.cpp:178] Use CPU.
...
I0520 16:06:14.553506 12424 caffe.cpp:222] Optimization Done.
~8 min

4个线程

I0520 16:06:44.634735 12446 caffe.cpp:178] Use CPU.
...
I0520 16:13:15.904394 12446 caffe.cpp:222] Optimization Done.
~6.5 min

ps -T -p <PID>给我:

export OPENBLAS_NUM_THREADS=1 6个线程

export OPENBLAS_NUM_THREADS=4 9个线程

似乎openblas可以工作，但这取决于网络体系结构吗?

Seems openblas works, but it depends on network architecture?

Caffe还将BLAS用于转换层.

推荐答案

我在CentOS 7.3上使用了intel分支，并且在训练caffenet示例时能够在Broadwell上看到2500％的CPU使用率.以下是我用来构建工具的步骤:

I'm using the intel branch on CentOS 7.3 and able to see 2500% CPU usage on my Broadwell when training caffenet example. The following is the steps I used to build the tools:

git clone https://github.com/BVLC/caffe.git caffe_intel
cd caffe_intel
git branch -r
git checkout intel
cp Makefile.config.example Makefile.config
  # mkl will be automatically download once make is run
  # Edit Makefile.config in the following lines
  PYTHON_LIB := /usr/lib64
  LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib64 /usr/lib   
yum install cmake # will be used to build mkl as a sub-routine of make
make -j32 all
make pycaffe
make distribute
cd distribute/lib
ln -s ../../external/mkldnn/install/lib/libmkldnn.so .

并将caffe_intel/distribute/bin放在$ PATH中，将caffe_intel/distribute/lib放在$ LD_LIBRARY_PATH中.另外，通过在prototxt文件的开头添加以下行来启用mkl库

And put caffe_intel/distribute/bin in $PATH, caffe_intel/distribute/lib in $LD_LIBRARY_PATH. Also, enable mkl library by adding the following line at the beginning of prototxt file(s)

engine: "MKL2017"

这篇关于Caffe多CPU构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！