Vowpal Wabbit中带有vw-hypersearch的多维超参数搜索 [英] Multidimensional hyperparameter search with vw-hypersearch in Vowpal Wabbit

问题描述

vw-hypersearch 是Vowpal Wabbit包装器，用于在大众汽车模型​​中优化超参数:正则化率，学习率和衰减，小批量，自举大小等.在教程有以下示例:

vw-hypersearch is the Vowpal Wabbit wrapper intended to optimize hyperparameters in vw models: regularization rates, learning rates and decays, minibatches, bootstrap sizes etc. In the tutorial for vw-hypersearch there is a following example:

vw-hypersearch  1e-10  5e-4  vw  --l1 %  train.dat

此处%表示要优化的参数，1e-10 5e-4是要搜索的时间间隔的上限和下限.该库使用黄金分割搜索方法来最大程度地减少迭代次数.

Here % means the parameter to be optimized, 1e-10 5e-4 are the lower and upper bounds for the interval over which to search. The library uses golden section search method to minimize the number of iterations.

但是，如果我想搜索多个超参数怎么办?从此 github问题的讨论中，我得到的提示可能是没有多维搜索方法在大众中实现.因此，唯一的出路是编写自己的特定于任务的优化器.我说的对吗?

But what if I want to search over multiple hyperparameters? From the sources like this github issue discussion, I get a hint that possibly no multidimentional search methods are realized in vw. Thus, the only way out is to write one's own task-specific optimizers. Am I right?

推荐答案

现在，可以使用资源库中位于/vowpal_wabbit/utl/的模块vw-hyperopt.py来完成此操作.

Now this can be done with the module vw-hyperopt.py that lives at /vowpal_wabbit/utl/ in the repository.

在此处查看我的请求: https://github.com/JohnLangford/vowpal_wabbit/pull /867

See my pull-request here: https://github.com/JohnLangford/vowpal_wabbit/pull/867

在不久的将来，这将得到更好的记录.

In the near future this will be better documented.

这篇关于Vowpal Wabbit中带有vw-hypersearch的多维超参数搜索的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！