我怎样才能告诉H2O深度学习网格具有AUC而不是残余偏差 [英] How can I tell h2o deep learning grid to have AUC instead of residual deviance

问题描述

我想通过寻找AUC或准确性来衡量模型的性能.在网格搜索中，我得到的结果是residual deviance，如何告诉h2o深度学习网格具有AUC而不是残余偏差，并将结果呈现为像下面所附的一样?

I would like to measure models performance by looking for AUC or Accuracy. In the grid search I get results with residual deviance,how can I tell h2o deep learning grid to have AUC instead of residual deviance and present the results as atable like the one attached below ?

train <- read.table(text = "target birds    wolfs     snakes
                              0        9         7 a
                              0        8         4 b
                              1        2         8 c
                              1        2         3 a
                              1        8         3 a
                              0        1         2 a
                              0        7         1 b
                              0        1         5 c
                              1        9         7 c
                              1        8         7 c
                              0        2         7 b
                              1        2         3 b
                              1        6         3 c
                              0        1         1 a
                              0        3         9 a
                              1        1         1 b ",header = TRUE)
trainHex <- as.h2o(train)

g <- h2o.grid("deeplearning",
              hyper_params = list(
                  seed = c(123456789,12345678,1234567),
                  activation = c("Rectifier", "Tanh", "TanhWithDropout", "RectifierWithDropout", "Maxout", "MaxoutWithDropout")
              ),
              reproducible = TRUE,
              x = 2:4,
              y = 1,
              training_frame = trainHex,
              validation_frame = trainHex,
              epochs = 50,
              )
g
model_ids <- g@summary_table
model_ids<-as.data.frame(model_ids)

我得到的结果表:

     Hyper-Parameter Search Summary: ordered by increasing residual_deviance
             activation      seed                                                  model_ids   residual_deviance
1                Maxout  12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_10 0.07243775676256235
2                Maxout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_16 0.10060885040861599
3     MaxoutWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_5  0.1706496158406441
4                Maxout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_4 0.17243125875659948
5                  Tanh 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_1 0.18326527198894926
6                  Tanh  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_7 0.18763395264761593
7                  Tanh   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_13 0.18791531211136187
8       TanhWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_2 0.19808063817007837
9       TanhWithDropout  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_8 0.19815190962052193
10      TanhWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_14 0.19832946889767458
11            Rectifier 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_0 0.20679125165086842
12    MaxoutWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_17 0.21971759565380736
13 RectifierWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_3 0.22337599298253263
14    MaxoutWithDropout  12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_11 0.22440661112729862
15 RectifierWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_15  0.2284671685474275
16 RectifierWithDropout  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_9 0.23163744415703522
17            Rectifier   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_12  0.2516917276707789
18            Rectifier  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_6  0.2642221616447725

推荐答案

您可以使用h2o.getGrid()进行此操作.接下来是您的示例代码:

You can do this with h2o.getGrid(). Following on from your example code:

g_rmse <- h2o.getGrid(g@grid_id, "rmse")
g_rmse  #Output it

我在那里选择了根MSE. AUC无法用于您的样本数据:它必须是二项式分类，并且您正在进行回归.

I've chosen root-MSE there. AUC is not available for your sample data: it has to be a binomial classification, and you are doing a regression.

您进行回归的原因是您的y包含0和1，因此H2O猜测它是数字.将其上传到H2O之后，您需要在该列上使用as.factor().

The reason you are doing a regression is your y contains 0 and 1, so H2O has guessed it is numeric. You need to use as.factor() on that column, just after uploading it into H2O.

train <-  ...
trainHex <- as.h2o(train)
trainHex[,1] = as.factor(trainHex[,1])  #Add this

g <- ...

然后您可以执行以下操作:

Then you can do this:

g_auc <- h2o.getGrid(g@grid_id, "auc", decreasing = TRUE)
g_auc

我将其设置为decreasing=TRUE，以便最好的AUC位于顶部.

I've set it to decreasing=TRUE so that the best AUC is at the top.

这篇关于我怎样才能告诉H2O深度学习网格具有AUC而不是残余偏差的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！