在H2O中的多个测试集上的ROC(Python) [英] ROC on multiple test sets in h2o (python)

问题描述

我有一个用例，我认为它很简单，但是找不到用h2o做到这一点的方法.我以为你可能知道.

I had a use-case that I thought was really simple but couldn't find a way to do it with h2o. I thought you might know.

我想训练我的模型一次，然后在几个不同的测试集(例如，一个验证集和一个测试集，尽管实际上我有两个以上)上评估其ROC，而不必重新训练模型.我现在知道的方式要求每次都重新训练模型:

I want to train my model once, and then evaluate its ROC on a few different test sets (e.g. a validation set and a test set, though in reality I have more than 2) without having to retrain the model. The way I know to do it now requires retraining the model each time:

train, valid, test = fr.split_frame([0.2, 0.25], seed=1234)
rf_v1 = H2ORandomForestEstimator( ... )
rf_v1.train(features, var_y, training_frame=train, validation_frame=valid)
roc = rf_v1.roc(valid=1)

rf_v1.train(features, var_y, training_frame=train, validation_frame=test) # training again with the same training set - can I avoid this?
roc2 = rf_v1.roc(valid=1)

我还可以使用model_performance()，它为我提供了任意测试集的一些指标，而无需重新训练，但ROC却没有.有没有办法使ROC脱离H2OModelMetrics对象?

I can also use model_performance(), which gives me some metrics on an arbitrary test set without retraining, but not the ROC. Is there a way to get the ROC out of the H2OModelMetrics object?

谢谢！

推荐答案

您可以使用h2o流检查模型性能.只需转到: http://localhost:54321/flow/index.html (如果您更改了默认端口，在链接中将其更改)；在单元格中输入"getModel"rf_v1"，它将向您显示流中多个单元格中模型的所有度量.非常方便. 如果您使用的是Python，则可以在IDE中找到这样的性能:

You can use the h2o flow to inspect the model performance. Simply go to: http://localhost:54321/flow/index.html (if you changed the default port change it in the link); type "getModel "rf_v1"" in a cell and it will show you all the measurements of the model in multiple cells in the flow. It's quite handy. If you are using Python, you can find the performance in your IDE like this:

rf_perf1 = rf_v1.model_performance(test)

然后按如下所示打印ROC:

and then print the ROC like this:

print (rf_perf1.auc())

这篇关于在H2O中的多个测试集上的ROC(Python)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！