SVM和神经网络 [英] SVM and Neural Network

问题描述

SVM和神经网络有什么区别? 线性svm是否与NN相同，对于非线性可分离问题，NN使用添加隐藏层，而SVM使用更改空间尺寸吗?

What is difference between SVM and Neural Network? Is it true that linear svm is same NN, and for non-linear separable problems, NN uses adding hidden layers and SVM uses changing space dimensions?

推荐答案

此问题分为两部分.第一部分是通过这些方法学习的功能形式是什么?"对于NN和SVM，这通常是相同的.例如，单个隐藏层神经网络使用与SVM完全相同的模型形式.那就是:

There are two parts to this question. The first part is "what is the form of function learned by these methods?" For NN and SVM this is typically the same. For example, a single hidden layer neural network uses exactly the same form of model as an SVM. That is:

给定输入向量x，输出为: 输出(x)= sum_over_all_i weight_i *非线性函数_i(x)

Given an input vector x, the output is: output(x) = sum_over_all_i weight_i * nonlinear_function_i(x)

通常，非线性函数也将具有一些参数.因此，这些方法需要了解应使用多少个非线性函数，它们的参数是什么以及所有weight_i权重的值应该是什么.

Generally the nonlinear functions will also have some parameters. So these methods need to learn how many nonlinear functions should be used, what their parameters are, and what the value of all the weight_i weights should be.

因此，SVM和NN之间的区别在于它们如何决定应将这些参数设置为什么.通常，当有人说他们正在使用神经网络时，这意味着他们试图找到相对于一组训练示例而言最小化均方预测误差的参数.他们几乎总是会使用随机梯度下降优化算法来执行此操作.另一方面，SVM尝试最小化训练误差和某种程度的假设复杂性".因此，他们将找到一组适合数据但在某种意义上也是简单"的参数.您可以将其视为Occam的机器学习剃刀.支持SVM的最常见优化算法是顺序最小优化.

Therefore, the difference between a SVM and a NN is in how they decide what these parameters should be set to. Usually when someone says they are using a neural network they mean they are trying to find the parameters which minimize the mean squared prediction error with respect to a set of training examples. They will also almost always be using the stochastic gradient descent optimization algorithm to do this. SVM's on the other hand try to minimize both training error and some measure of "hypothesis complexity". So they will find a set of parameters that fits the data but also is "simple" in some sense. You can think of it like Occam's razor for machine learning. The most common optimization algorithm used with SVMs is sequential minimal optimization.

这两种方法之间的另一个大区别是，当使用NN实现的方式时，不能保证随机梯度下降法可以找到最佳的参数集.但是，任何体面的SVM实现都将找到最佳的参数集.人们喜欢说神经网络陷入了局部最小值，而SVM却没有.

Another big difference between the two methods is that stochastic gradient descent isn't guaranteed to find the optimal set of parameters when used the way NN implementations employ it. However, any decent SVM implementation is going to find the optimal set of parameters. People like to say that neural networks get stuck in a local minima while SVMs don't.

这篇关于SVM和神经网络的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！