神经网络的Adaboost [英] Adaboost with neural networks
问题描述
我为一个项目实现了Adaboost，但不确定我是否正确理解了adaboost.这是我实施的方法，请告诉我它是否正确.
I implemented Adaboost for a project, but I'm not sure if I've understood adaboost correctly. Here's what I implemented, please let me know if it is a correct interpretation.
- 我的弱分类器是8个不同的神经网络.经过全面训练后，这些方法中的每一项都可以预测约70％的准确性. 我充分训练了所有这些网络，并在训练集上收集了它们的预测；所以我在训练集上有8个预测向量.
- My weak classifiers are 8 different neural networks. Each of these predict with around 70% accuracy after full training.
- I train all these networks fully, and collect their predictions on the training set ; so I have 8 vectors of predictions on the training set.
现在我使用adaboost.我对adaboost的解释是，它将找到最终的分类器，作为我在上面训练过的分类器的加权平均值，其作用是找到这些权重.因此，对于每个训练示例，我都有8个预测，并且使用adaboost权重将它们组合在一起.请注意，通过这种解释，弱分类器在adaboost迭代过程中不会重新训练，只对权重进行更新.但是，更新后的权重实际上会在每次迭代中创建新的分类器.
Now I use adaboost. My interpretation of adaboost is that it will find a final classifier as a weighted average of the classifiers I have trained above, and its role is to find these weights. So, for every training example I have 8 predictions, and I'm combining them using adaboost weights. Note that with this interpretation, the weak classifiers are not retrained during the adaboost iterations, only the weights are updated. But the updated weights in effect create new classifiers in each iteration.
这是伪代码:
all_alphas = []
all_classifier_indices = []
initialize all training example weights to 1/(num of examples)
compute error for all 8 networks on the training set
for i in 1 to T:
find the classifier with lowest weighted error.
compute the weights (alpha) according to the Adaboost confidence formula
Update the weight distribution, according to the weight update formula in Adaboost.
all_alphas.append(alpha)
all_classifier_indices.append(selected_classifier)
在T迭代之后，有T个alpha和T个分类器索引；这些T分类器索引将指向8个神经网络预测向量之一.
After T iterations, there are T alphas and T classifier indices ; these T classifier indices will point to one of the 8 neural net prediction vectors.
然后在测试集中，对于每个示例，我通过对alpha*classifier求和来预测.
Then on the test set, for every example, I predict by summing over alpha*classifier .
我想在神经网络中使用adaboost，但我认为我误解了adaboost算法.
I want to use adaboost with neural networks, but I think I've misinterpreted the adaboost algorithm wrong..
推荐答案
提升摘要:
1-使用训练数据训练您的第一个弱分类器
1- Train your first weak classifier by using the training data
2-经过第1次训练的分类器在某些样本上犯了错误，并对其他样本进行了正确的分类.增加错误分类的样本的权重，减少正确分类的样本的权重.使用这些权重重新训练您的分类器，以获得第二分类器.
2- The 1st trained classifier makes mistake on some samples and correctly classifies others. Increase the weight of the wrongly classified samples and decrease the weight of correct ones. Retrain your classifier with these weights to get your 2nd classifier.
对于您而言，您首先必须使用这些更新后的权重从数据中重新采样并进行替换，创建新的训练数据，然后根据这些新数据训练您的分类器.
In your case, you first have to resample with replacement from your data with these updated weights, create a new training data and then train your classifier over these new data.
3-重复第二步T次，并在每轮结束时，根据公式计算分类器的alpha权重. 4-最终分类器是T分类器决策的加权和.
3- Repeat the 2nd step T times and at the end of each round, calculate the alpha weight for the classifier according to the formula. 4- The final classifier is the weighted sum of the decisions of the T classifiers.
希望从此解释中可以清楚地看出您做错了一点.无需使用新数据集来重新训练网络，而是可以在整个原始数据集中对其进行训练.实际上，您正在使用随机森林类型分类器(除了您使用的是NN而不是决策树)集成.
It is hopefully clear from this explanation that you have done it abit wrongly. Instead of retrain your network with the new data set, you trained them all over the original dataset. In fact you are kind of using random forest type classifier (except that you are using NN instead of decision trees) ensemble.
PS:不能保证增强会提高准确性.实际上，到目前为止，我所知道的所有增强方法都无法提高作为弱学习者的NN的准确性(原因是因为增强的工作方式并且需要进行更长时间的讨论).
PS: There is no guarantee that boosting increases the accuracy. In fact, so far all the boosting methods that I'm aware of were unsuccessful to improve the accuracy with NN as weak learners (The reason is because of the way that boosting works and needs a lengthier discussion).
这篇关于神经网络的Adaboost的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
