为什么不通过回归来进行分类? [英] Why Is It Not To Approach Classification Through Regression?

问题描述

机器学习材料说，通过回归来解决分类问题是一个坏主意.但是我认为始终可以进行连续回归以拟合数据，并截断连续预测以产生离散分类.那么，为什么这是一个坏主意呢?

A machine learning material said that it's a bad idea to approach classification problem through regression. But I think it's always possible to do a continuous regression to fit the data and truncate the continuous prediction to yield discrete classification. So why it's a bad idea?

推荐答案

如果要进行分类，则要优化与分类错误有关的内容.您只关心预测合适的班级.在进行回归时，您希望最小化预测值与实际值之间的某种程度的失真.均方误差是回归的常用惩罚函数.

If you are doing classification, you want to optimize something related to misclassifications. You only care about predicting the right class. When you are doing regression, you want to minimize some measure of distortion between the prediction and the actual value. Mean squared error is a common penalty function for regression.

想象一下，优化最终将要进行分类的回归器的参数.出现一个示例，该示例显然是1类，但其标签非常非常大.为了最大程度地减少此示例的损失，您必须进行大量调整以使此示例的预测极端.但是，现在您的分类边界移动了很多，从而损害了分类的准确性.当您不需要时，您会得到过度补偿.

Imagine optimizing the parameters of your regressor that is eventually going to do classification. In comes a an example that is obviously class 1, but whose label is very, very large. In order to minimize the loss on this example, you have to shift your weights a lot to make the prediction extreme for this example. However, now your classification border just moved a lot, hurting your classification accuracy. You over-compensated when you didn't need to.

您可以根据权重预测示例的方式来移动权重的数量来查看该图.

You can view this graph as the amount you'll move your weights as a function of how you mis-predicted an example.

这里的大多数损失函数是误分类损失的上限.优化分类错误上限的模型可以很好地进行分类.使用回归进行分类类似于选择平方误差损失，并且实质上是错误地表示您要优化的内容.即使分类变得越来越有信心，并且良好的分类损失函数全都为0或在那儿出现，这也对应于平方误差损失朝图的右侧向上移位.

Most of the loss functions here are upperbounds on the misclassification loss. Models that optimize upperbounds on misclassification do classification well. Using regression for classification is akin to picking the squared error loss, and essentially mis-representing what you want to optimize. This corresponds to the upward shift toward the right side of the graph in the loss for squared error, even as the classification is becoming more and more confident, and the good classification loss functions are all either 0 or going there.

图片摘自出色的统计学习理论的要素.

这篇关于为什么不通过回归来进行分类?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！