H2O模型不适合驱动程序节点的内存错误 [英] h2o model not fit in driver node's memory error

问题描述

我在H2O中通过R代码运行了GBM模型，但出现了以下错误.相同的代码运行了好几个星期.想知道这是H2O方面的错误还是在用户系统上进行的配置?

I ran GBM model through R code in H2O and got below error. The same code was running fine a couple of weeks. Wondering if this is H2O side error Or configuration on the user system?

water.exceptions.H2OModelBuilderIllegalArgumentException:GBM模型的非法参数:gbm-2017-04-18-15-29-53.详细信息:字段上的ERRR:_ntrees:树模型将不适合驱动程序节点的内存(每棵树23.2 MB x 1000> 3.32 GB)-尝试减少ntree和/或max_depth或增加min_rows！

water.exceptions.H2OModelBuilderIllegalArgumentException: Illegal argument(s) for GBM model: gbm-2017-04-18-15-29-53. Details: ERRR on field: _ntrees: The tree model will not fit in the driver node's memory (23.2 MB per tree x 1000 > 3.32 GB) - try decreasing ntrees and/or max_depth or increasing min_rows!

推荐答案

对我有用的解决方法是在初始化H2O时设置 both 和最小内存大小.例如:

The fix that worked for me was to set both the min and max memory sizes when initializing H2O. For example:

当未指定最小或最大内存大小时，此操作将失败:

This fails when not specifying either min or max memory size:

localH2O <- h2o.init(ip='localhost', nthreads=-1)

INFO: Java heap totalMemory: 1.92 GB
INFO: Java heap maxMemory: 26.67 GB
INFO: Java version: Java 1.8.0_121 (from Oracle Corporation)
INFO: JVM launch parameters: [-ea]
INFO: OS version: Linux 3.10.0-327.el7.x86_64 (amd64)
INFO: Machine physical memory: 1.476 TB

仅指定最大内存大小时失败:

This fails when specifying only max memory size:

localH2O <- h2o.init(ip='localhost', nthreads=-1,
                     max_mem_size='200G')

INFO: Java availableProcessors: 64
INFO: Java heap totalMemory: 1.92 GB
INFO: Java heap maxMemory: 177.78 GB
INFO: Java version: Java 1.8.0_121 (from Oracle Corporation)
INFO: JVM launch parameters: [-Xmx200G, -ea]
INFO: OS version: Linux 3.10.0-327.el7.x86_64 (amd64)
INFO: Machine physical memory: 1.476 TB

当同时指定最小存储大小和最大存储大小时，这将成功:

This is successful when specifying both min and max memory sizes:

localH2O <- h2o.init(ip='localhost', nthreads=-1,
                     min_mem_size='100G', max_mem_size='200G')

INFO: Java availableProcessors: 64
INFO: Java heap totalMemory: 95.83 GB
INFO: Java heap maxMemory: 177.78 GB
INFO: Java version: Java 1.8.0_121 (from Oracle Corporation)
INFO: JVM launch parameters: [-Xms100G, -Xmx200G, -ea]
INFO: OS version: Linux 3.10.0-327.el7.x86_64 (amd64)
INFO: Machine physical memory: 1.476 TB

这篇关于H2O模型不适合驱动程序节点的内存错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！