在python中使用hypot包在GridSearch函数中指定评分指标 [英] specify scoring metric in GridSearch function with hypopt package in python

问题描述

我正在使用来自hypopt包的Gridsearch函数来使用指定的验证集进行我的超参数搜索.分类的默认指标似乎是准确性(不是很确定).在这里，我想使用F1分数作为指标.我不知道应该在哪里指定指标.我看了看文档，但有点困惑.

I'm using Gridsearch function from hypopt package to do my hyperparameter searching using specified validation set. The default metric for classification seems to be accuracy (not very sure). Here I want to use F1 score as the metric. I do not know where I should specify the metric. I looked at the documentation but kind of confused.

熟悉hypot软件包的人知道我该怎么做吗?提前非常感谢.

Does anyone who are familiar with hypopt package know how I can do this? Thanks a lot in advance.

from hypopt import GridSearch

log_reg_params = {"penalty": ['l1'], 'C': [0.001, 0.01]}
opt = GridSearch(model=LogisticRegression())
opt.fit(X_train, y_train, log_reg_params, X_val, y_val)

推荐答案

hypopt包的默认度量标准是score()函数(对于所使用的任何模型)，因此在您的情况下，默认值为LogisticRegression().score()准确.

The default metric of the hypopt package is the the score() function for whatever model you use, so in your case it is LogisticRegression().score() which defaults to accuracy.

如果通过pip install hypopt --upgrade将hypot软件包升级到1.0.8版，则可以在GridSearch.fit()的scoring参数中指定您选择的任何度量标准，例如fit(scoring='f1').这是一个基于您的代码的简单工作示例，该代码使用F1指标:

If you upgrade the hypopt package to version 1.0.8 via pip install hypopt --upgrade, you can specify any metric of your choosing in the scoring parameter of GridSearch.fit(), for example, fit(scoring='f1'). Here is a simple working example based on your code that uses the F1 metric:

from hypopt import GridSearch

param_grid = {"penalty": ['l1'], 'C': [0.001, 0.01]}
opt = GridSearch(model=LogisticRegression(), param_grid = param_grid)
# This will use f1 score as the scoring metric that you optimize.
opt.fit(X_train, y_train, X_val, y_val, scoring='f1')

hypopt支持大多数sklearn支持的评分功能.

hypopt supports most any scoring function that sklearn supports.

• 对于分类，hypopt支持以下度量标准(作为字符串):'准确性'，'brier_score_loss'，'average_precision'，'f1'，'f1_micro'，'f1_macro'，'f1_weighted'，'neg_log_loss'，'精度"，调用"或"roc_auc".
• 对于回归，hypopt支持:"explained_variance"，"neg_mean_absolute_error"，"neg_mean_squared_error"，"neg_mean_squared_log_error"，"neg_median_absolute_error"，"r2".

• For classification, hypopt supports these metrics (as strings): 'accuracy', 'brier_score_loss', 'average_precision', 'f1', 'f1_micro', 'f1_macro', 'f1_weighted', 'neg_log_loss', 'precision', 'recall', or 'roc_auc'.
• For regression, hypopt supports: "explained_variance", "neg_mean_absolute_error", "neg_mean_squared_error", "neg_mean_squared_log_error", "neg_median_absolute_error", "r2".

您还可以通过将其包装到这样的对象中来创建自己的指标your_custom_score_func(y_true, y_pred):

You can also create your own metric your_custom_score_func(y_true, y_pred) by wrapping it into an object like this:

from sklearn.metrics import make_scorer
scorer = make_scorer(your_custom_score_func)
opt.fit(X_train, y_train, X_val, y_val, scoring=scorer)

您可以在hypopt.GridSearch.fit()文档字符串中了解更多信息:

You can learn more in the hypopt.GridSearch.fit() docstring here:

• https://github.com/cgnorthcutt/hypopt /blob/master/hypopt/model_selection.py#L240

您可以在此处了解有关创建自己的自定义评分指标的更多信息:

You can learn more about creating your own custom scoring metrics here:

• 示例: https://github.com/cgnorthcutt /hypopt/blob/master/tests/test_core.py#L371
• 源代码: http://scikit-learn. org/stable/modules/generated/sklearn.metrics.make_scorer.html

• Example: https://github.com/cgnorthcutt/hypopt/blob/master/tests/test_core.py#L371
• Source code: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

这篇关于在python中使用hypot包在GridSearch函数中指定评分指标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！