xgboost:线性升压器gblinear中使用了哪些参数? [英] xgboost: which parameters are used in the linear booster gblinear?

问题描述

在网络上看，我仍然对线性助推器gblinear到底是什么感到困惑，而我不是

Looking on the web I am still a confused about what the linear booster gblinear precisely is and I am not alone.

在文档之后，它只有3个参数lambda，lambda_ bias和alpha-也许应该说其他参数".

Following the documentation it only has 3 parameters lambda,lambda_bias and alpha - maybe it should say "additional parameters".

如果我正确理解这一点，那么线性助推器会(而不是标准)进行线性助推(带有正则化). 在这种情况下，我只能理解上面的3个参数和eta(提升率). 这也是 github 上的描述方式

If I understand this correctly then the linear booster does (rather standard) linear boosting (with regularization). In this context I can only make sense of the 3 parameters above and eta (the boosting rate). That's also how it is described on github.

尽管如此，我仍然看到树参数gamma，max_depth和min_child_weight对算法也有影响.

Nevertheless I see that tree parameters gamma,max_depth and min_child_weight also have an impact on the algorithm.

怎么可能?网络上的任何地方都对线性助力器有一个完全清晰的描述吗?

How can this be? Is there a totally clear description of the linear booster anywhere on the web?

请参阅我的示例:

library(xgboost)

data(agaricus.train, package='xgboost')
data(agaricus.test, package='xgboost')
train <- agaricus.train
test <- agaricus.test

然后设置

set.seed(100)
model <- xgboost(data = train$data, label = train$label, nrounds = 5, 
                 objective = "binary:logistic", 
                 params = list(booster = "gblinear", eta = 0.5, lambda = 1, lambda_bias = 1,gamma = 2,
                               early_stopping_rounds = 3))

给予

> [1]   train-error:0.018271  [2]   train-error:0.003071 
> [3]   train-error:0.001075  [4]   train-error:0.001075 
> [5]   train-error:0.000614

而gamma=1

set.seed(100)
model <- xgboost(data = train$data, label = train$label, nrounds = 5, 
                 objective = "binary:logistic", 
                 params = list(booster = "gblinear", eta = 0.5, lambda = 1, lambda_bias = 1,gamma = 1,
                               early_stopping_rounds = 3))

导致

> [1]   train-error:0.013051  [2]   train-error:0.001842 
> [3]   train-error:0.001075  [4]   train-error:0.001075 
> [5]   train-error:0.001075

这是另一个路径".

与max_depth类似:

set.seed(100)
model <- xgboost(data = train$data, label = train$label, nrounds = 5, 
                 objective = "binary:logistic", 
                 params = list(booster = "gblinear", eta = 0.5, lambda = 1, lambda_bias = 1, max_depth = 3,
                               early_stopping_rounds = 3))

> [1]   train-error:0.016122  [2]   train-error:0.002764 
> [3]   train-error:0.001075  [4]   train-error:0.001075 
> [5]   train-error:0.000768

和

set.seed(100)
model <- xgboost(data = train$data, label = train$label, nrounds = 10, 
                 objective = "binary:logistic", 
                 params = list(booster = "gblinear", eta = 0.5, lambda = 1, lambda_bias = 1, max_depth = 4,
                               early_stopping_rounds = 3))

> [1]   train-error:0.014740  [2]   train-error:0.004453 
> [3]   train-error:0.001228  [4]   train-error:0.000921 
> [5]   train-error:0.000614

推荐答案

我也可以在运行gblinear之间做一些下蹲，观察结果几乎每次都改变，并声称下蹲对算法有影响:)

I may as well do some squats between running the gblinear, observe the results change almost every time, and claim that doing squats has an impact on the algorithm :)

严格地说，gblinear当前使用的算法不是您的相当标准的线性提升".造成随机性的原因是在每次迭代期间更新梯度时使用无锁并行化("hogwild").设置种子不会影响任何事情.并且只有在运行单线程(nthread=1)时才能获得始终如一的可重复结果.我还建议不要使用默认的nthread设置来运行它，因为它会使用最大数量的OpenMP线程，因为在许多系统上，由于线程拥塞，它会导致速度大大降低. nthread不得高于物理内核的数量.

In all seriousness, the algorithm that gblinear currently uses is not your "rather standard linear boosting". The thing responsible for the stochasticity is the use of lock-free parallelization ('hogwild') while updating the gradients during each iteration. Setting the seed doesn't affect anything; and you would only get consistently reproducible results when running single-threaded (nthread=1). I would also advise against running it with the default nthread setting which uses maximum possible number of OpenMP threads, as on many systems it would result in much slower speed due to thread congestion. The nthread needs to be not higher than the number of physical cores.

这种自由随机性在某些情况下可能会提高预测性能.但是，优点通常不会超过缺点.在某个时候，我将提交拉取请求，其中包含用于确定性并行化的选项，以及用于在每个提升回合中对功能选择进行一些其他控制的选项.

This free stochasticity might improve predictive performance in some situations. However, the pros frequently don't outweigh cons. At some point, I will submit a pull request with an option for deterministic parallelization and an option for some additional control over feature selection at each boosting round.

有关特定于增强训练的所有可用参数的真实情况，请参考

For ground truth on all the available parameters that are specific to booster training, refer to the source of struct GBLinearTrainParam for gblinear and to the source of struct TrainParam for gbtree.

这篇关于xgboost:线性升压器gblinear中使用了哪些参数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！