libsvm收缩启发式 [英] libsvm Shrinking Heuristics

问题描述

我正在C-SVC模式下使用libsvm，其阶数为2的多项式内核，并且需要训练多个SVM.在培训期间，对于我训练的某些SVM，我会收到这些警告中的一个甚至两个:

I'm using libsvm in C-SVC mode with a polynomial kernel of degree 2 and I'm required to train multiple SVMs. During training, I am getting either one or even both of these warnings for some of the SVMs that I train:

WARNING: using -h 0 may be faster
*
WARNING: reaching max number of iterations
optimization finished, #iter = 10000000

我找到了h参数的描述:

-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)

并且我尝试阅读libsvm文档中的解释，但是对我来说有点太高了.谁能提供一个外行的解释，也许有一些建议，例如设置此项会因为……而有所帮助?另外，了解一下是否可以通过为我训练的所有SVM设置此参数，是否会对那些未明确发出此警告的SVM的准确性产生负面影响.

and I've tried to read the explanation from the libsvm documentation, but it's a bit too high level for me. Can anyone please provide a layman's explanation and, perhaps, some suggestions like setting this would be beneficial because...? Also, it would be helpful to know if by setting this parameter for all the SVMs that I train, might produce negative impact on accuracy for those SVMs that do not explicitly give this warning.

我不确定该如何处理其他警告.

I'm not sure what to make of the other warning.

仅提供更多细节:我的训练集具有10个属性(特征)，并且包含5000个向量.

Just to give more details: my training sets have 10 attributes (features) and they consist of 5000 vectors.

更新:

万一其他人获得了达到最大迭代次数"，这似乎是由数值稳定性问题引起的.同样，这将导致非常缓慢的训练时间.多项式内核确实受益于使用交叉验证技术来确定正则化的最佳值(C参数)，对于多项式内核，对我而言，它有助于将其保持在小于8的水平.此外，如果内核是不均匀的\ sum(\ gamma x_i s_i + coef0)^ d(对不起，SO不支持LaTeX)，其中coef0！= 0，则可以使用针对gamma和C的网格搜索技术来实现交叉验证，因为在这种情况下在这种情况下，伽玛的默认值(1/number_of_features)可能不是最佳选择.不过，从我的实验来看，您可能不希望伽玛太大，因为它会引起数值问题(我正在尝试将其最大值设为8).

In case anybody else is getting the "reaching max number of iterations", it seems to be caused by numeric stability issues. Also, this will produce a very slow training time. Polynomial kernels do benefit from using cross-validation techniques to determine the best value for regularization (the C parameter), and, in the case of polynomial kernels, for me it helped to keep it smaller than 8. Also, if the kernel is inhomogeneous \sum(\gamma x_i s_i + coef0)^d (sorry, LaTeX is not supported on SO), where coef0 != 0, then cross validation can be implemented with a grid search technique for both gamma and C, since, in this case, the default value for gamma (1 / number_of_features) might not be the best choice. Still, from my experiments, you probably do not want gamma to be too big, since it will cause numeric issues (I am trying a maximum value of 8 for it).

要想进一步了解gamma和C的可能值，应尝试在grid.py中戳入.

For further inspiration on the possible values for gamma and C one should try poking in grid.py.

推荐答案

缩小的启发式方法可以加快优化速度.就像在FAQ中说的那样，它们有时会有所帮助，有时却没有帮助.我认为这是运行时的问题，而不是收敛的问题.

The shrinking heuristics are there to speed up the optimization. As it says in the FAQ, they sometimes help, and sometimes they do not. I believe it's a matter of runtime, rather than convergence.

尽管如此，优化达到了最大迭代次数这一事实很有趣.您可能需要考虑公差(成本参数)，或查看引起此公差的各个问题.数据集很大吗?

The fact that the optimization reaches the maximum number of iterations is interesting, though. You might want to play with the tolerance (cost parameter), or have a look at the individual problems that cause this. Are the datasets large?

这篇关于libsvm收缩启发式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！