Vowpal Wabbit的梯度提升 [英] Gradient boosting on Vowpal Wabbit

问题描述

是否有办法在使用Vowpal Wabbit进行回归分析时使用梯度增强?我使用Vowpal Wabbit随附的各种有用的技术.我想同时尝试梯度增强，但是我找不到在大众汽车上实现梯度增强的方法.

Is there a way to use gradient boosting on regression using Vowpal Wabbit? I use various techniques that come with Vowpal Wabbit that are helpful. I want to try gradient boosting along with that, but I can't find a way to implement gradient boosting on VW.

推荐答案

梯度增强的想法是一个整体模型是根据黑匣子弱模型构建的.您当然可以将VW用作黑匣子，但请注意，VW不提供决策树，而决策树是黑匣子弱势模型在提升方面的最受欢迎选择.一般而言，提升可以降低偏差(并增加方差)，因此您应确保大众模型的方差低(不过度拟合).参见偏差方差权衡.

The idea of gradient boosting is that an ensemble model is built from black-box weak models. You can surely use VW as the black box, but note that VW does not offer decision trees, which are the most popular choice for the black-box weak models in boosting. Boosting in general decreases bias (and increases variance), so you should make sure that the VW models have low variance (no overfitting). See bias-variance tradeoff.

与大众化相关的一些减少措施:

There are some reductions related to boosting and bagging in VW:

• --autolink N添加了具有多项式N的链接函数，这可以被视为一种简单的提升方法.
• --log_multi K是用于K级分类的在线增强算法.参见本文.您甚至可以将其用于二进制分类(K = 2)，但不能用于回归.
通过在线重要性重采样进行
• --bootstrap M M方式引导.使用--bs_type=vote进行分类，使用--bs_type=mean进行回归.请注意，这是装袋，而不是加强
.
• --boosting N(于2015-06-17添加)在N个弱学习者的在线推动下，请参阅理论论文

• --autolink N adds a link function with polynomial N, which can be considered a simple way of boosting.
• --log_multi K is an online boosting algorithm for K-class classification. See the paper. You can use it even for binary classification (K=2), but not for regression.
• --bootstrap M M-way bootstrap by online importance resampling. Use --bs_type=vote for classification and --bs_type=mean for regression. Note that this is bagging, not boosting.
• --boosting N (added on 2015-06-17) online boosting with N weak learners, see a theoretic paper

这篇关于Vowpal Wabbit的梯度提升的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！