没有CUDA，是否可以在H2O中建立Deep Water/TensorFlow模型 [英] Is it possible to build Deep Water/TensorFlow model in H2O without CUDA

问题描述

我的目标是将H2O与TensorFlow集成在一起，而无需在计算机上使用CUDA.

由于TensorFlow同时支持CPU和GPU执行，因此我希望在没有CUDA的情况下实现H2O/TensorFlow集成.但是我很困惑在

您还可以在FLOW单元格中将gpu设置为false，如下所示:

"gpu":false

但是，您的主要问题是没有后端(mxnet，tensorflow，caffe)可用于运行您的代码.我们确实使用mxnet测试了gpu标志设置.请尝试调查有关上述错误的更多信息.

My goal is to integrate H2O with TensorFlow without CUDA on a machine.

As TensorFlow supports both CPU and GPU execution, I expect H2O/TensorFlow integration to be possible without CUDA. But I'm pretty confused by mentioning of CUDA software in system specifications of Deep Water.

I've tried to build Deep Water/TensorFlow model in H2O Flow but failed. The steps I've performed:

1. Downloaded H2O standalone JAR;
2. Created data frame in H2O Flow as usual;
3. Tried to build a model with Deep Water and tensorflow chosen as an algorithm and backend respectively;
4. Got the following exception:

java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: No backend found. Cannot build a Deep Water model.
    at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:246)
    at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:193)
    at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:225)
    at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:127)
    at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:114)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:169)
    at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:107)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1220)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

So my questions are:

1. Is it possible to build Deep Water/TensorFlow model in H2O without CUDA at all?
2. If it is, what should I do to get it working? If it is not, are there other options to integrate H2O and TensorFlow without CUDA?

Update 1:

I've set the gpu parameter to false and tried to build model again with all possible backends. Both caffe and tensorflow produce the same stacktrace as shown above. mxnet also fails but with two different stacktraces.

mxnet (first attempt to build a model):

java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: null
    at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:246)
    at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:193)
    at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:225)
    at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:127)
    at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:114)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:169)
    at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:107)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1220)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

mxnet (subsequent attempts):

java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: Could not initialize class deepwater.backends.mxnet.MXNetBackend$MXNetLoader
    at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:246)
    at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:193)
    at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:225)
    at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:127)
    at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:114)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:169)
    at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:107)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1220)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

Update 2

Environment:

• SW: CentOS Linux release 7.3.1611 (Core), Java HotSpot 64-Bit Server VM (build 25.121-b13, mixed mode);
• HW: virtual machine running on Xeon CPU E5-2620 v4 with 4 cores and 8 GB RAM available. No physical GPU is available, lspci -vnn | grep VGA returns 00:0f.0 VGA compatible controller [0300]: VMware SVGA II Adapter [15ad:0405] (prog-if 00 [VGA controller])

I've cleared my /tmp directory and tried mxnet again. On the first attempt I've got new exception:

java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: /tmp/libmxnet.so: libcudart.so.8.0: cannot open shared object file: No such file or directory
    at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:246)
    at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:193)
    at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:225)
    at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:127)
    at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:114)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:169)
    at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:107)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1220)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

The file /tmp/libmxnet.so is present, its permissions are -rw-rw-r--.

解决方案

The answer to your first question is as below:

You sure can get to run deep water without GPU, it will be very slow. When you are using FLOW you could disable gpu setting as below (which is TRUE by default)

Also you can set gpu as false in the FLOW cell as below:

"gpu":false

However your main problem is that none of the backend (mxnet, tensorflow, caffe) was available to run your code. We did test gpu flag settings with mxnet for sure. Please try to investigate more about the error above.

这篇关于没有CUDA，是否可以在H2O中建立Deep Water/TensorFlow模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！