如何在R中的h2o.grid中调整hidden_​​dropout_ratios [英] How to tune hidden_dropout_ratios in h2o.grid in R

问题描述

我想在R中使用h2o调整带有缺失的神经网络.在这里，我为虹膜数据集提供了一个可重现的示例.我避免仅出于提高计算速度的目的而调整eta和epsiplon(即ADADELTA超参数).

I want to tune a neural network with dropout using h2o in R. Here I provide a reproducible example for the iris dataset. I'm avoiding to tune eta and epsiplon (i.e. ADADELTA hyper-parameters) with the only purpose of making computations faster.

require(h2o)
h2o.init()
data(iris)
iris = iris[sample(1:nrow(iris)), ]
irisTrain = as.h2o(iris[1:90, ])
irisValid = as.h2o(iris[91:120, ])
irisTest = as.h2o(iris[121:150, ])
hyper_params <- list(
    input_dropout_ratio = list(0, 0.15, 0.3),
    hidden_dropout_ratios = list(0, 0.15, 0.3, c(0,0), c(0.15,0.15),c(0.3,0.3)),
    hidden = list(64, c(32,32)))
grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
                training_frame = irisTrain, validation_frame = irisValid,
                hyper_params = hyper_params, adaptive_rate = TRUE,
                variable_importances = TRUE, epochs = 50, stopping_rounds=5,
                stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
                seed=1, reproducible=TRUE)

输出为:

Details: ERRR on field: _hidden_dropout_ratios: Must have 1 hidden layer dropout ratios.

问题出在hidden_dropout_ratios中.请注意，我将input_dropout_ratio和hidden_​​dropout_ratios的值包括为0，因为我也想测试激活函数而不会丢失.我知道我可以使用activation="Rectifier，但是我认为我的配置应该导致相同的结果.调整具有不同层数的体系结构时如何调整hidden_dropout_ratios?

The problem is in hidden_dropout_ratios. Note that I'm including 0 for input_dropout_ratio and hidden_dropout_ratios since I also want to test the activation function without dropout. I'm aware that I could use activation="Rectifier but I think that my configuration should lead to the same result. How do I tune hidden_dropout_ratios when tuning architectures with different numbers of layers?

尝试1:不成功，我没有调整hidden.

Attempt 1: Unsuccessful and I'm not tuning hidden.

hyper_params <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(c(0.3,0.3), c(0.5,0.5)),
    hidden = c(32,32))
ERRR on field: _hidden_dropout_ratios: Must have 1 hidden layer dropout ratios.

尝试2:成功，但是我没有调整hidden.

Attempt 2: Successful but I'm not tuning hidden.

hyper_params <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = c(0.3,0.3),
    hidden = c(32,32))

推荐答案

如果尝试hidden_​​dropout_ratios，则必须修复网格中隐藏层的数量.起初，我把多个网格合并在一起搞乱了.然后，当研究我的H2O书籍时，我看到有人提到，如果给网格起相同的名称，网格是如何自动组合的.

You have to fix the number of hidden layers in a grid, if experimenting with hidden_dropout_ratios. At first I messed around with combining multiple grids; then, when researching for my H2O book, I saw someone mention, in passing, how grids get combined automatically if you give them the same name.

因此，您仍然需要为每个隐藏层数调用h2o.grid()，但是最后它们都可以位于同一网格中.这是为此修改的示例:

So, you still need to call h2o.grid() for each number of hidden layers, but they can all be in the same grid at the end. Here is your example modified for that:

require(h2o)
h2o.init()
data(iris)
iris = iris[sample(1:nrow(iris)), ]
irisTrain = as.h2o(iris[1:90, ])
irisValid = as.h2o(iris[91:120, ])
irisTest = as.h2o(iris[121:150, ])

hyper_params1 <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(0, 0.15, 0.3),
    hidden = list(64)
    )

hyper_params2 <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(c(0,0), c(0.15,0.15),c(0.3,0.3)),
    hidden = list(c(32,32))
    )

grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
    grid_id = "stackoverflow",
    training_frame = irisTrain, validation_frame = irisValid,
    hyper_params = hyper_params1, adaptive_rate = TRUE,
    variable_importances = TRUE, epochs = 50, stopping_rounds=5,
    stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
    seed=1, reproducible=TRUE)

grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
    grid_id = "stackoverflow",
    training_frame = irisTrain, validation_frame = irisValid,
    hyper_params = hyper_params2, adaptive_rate = TRUE,
    variable_importances = TRUE, epochs = 50, stopping_rounds=5,
    stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
    seed=1, reproducible=TRUE)

当我打印网格时，我被提醒使用列表超参数(例如hidden或hidden_​​dropout_ratios)时，网格输出存在错误.您的代码是一个很好的独立示例，因此我现在将对其进行报告.同时，这是一个单行代码，用于显示与每个参数对应的超参数的值:

When I went to print the grid, I was reminded there is a bug with grid output when using list hyper-parameters, such as hidden or hidden_dropout_ratios. Your code is a nice self-contained example, so I'll report that now. In the meantime, here is a one-liner to show the values of the hyper-parameter corresponding to each:

sapply(models, function(m) c(
  paste(m@parameters$hidden, collapse = ","),
  paste(m@parameters$hidden_dropout_ratios, collapse=",")
  ))

哪个给:

     [,1]    [,2] [,3]        [,4]   [,5]      [,6] 
[1,] "32,32" "64" "32,32"     "64"   "32,32"   "64" 
[2,] "0,0"   "0"  "0.15,0.15" "0.15" "0.3,0.3" "0.3"

即隐藏的辍学总比没有好，总有很多.而且两个隐藏层比一个更好.

I.e. no hidden dropout is better than a little, which is better than a lot. And two hidden layers is better than one.

顺便说一句，

• input_dropout_ratio:控制输入层和第一个隐藏层之间的退出.可以独立于激活功能使用.
• hidden_dropout_ratios:控制每个隐藏层和下一层(下一个隐藏层或输出层)之间的缺失.如果指定，则必须指定"WithDropout"激活功能之一.

• input_dropout_ratio: controls dropout between input layer and the first hidden layer. Can be used independently of the activation function.
• hidden_dropout_ratios: controls dropout between each hidden layer and the next layer (which is either the next hidden layer, or the output layer). If specified, you must specify one of the "WithDropout" activation functions.

这篇关于如何在R中的h2o.grid中调整hidden_​​dropout_ratios的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！