在没有PCA的多类svm中找到正确的功能 [英] Find right features in multiclass svm without PCA

问题描述

我要用一个多类svm(一对一)对用户进行分类，共3个类.以二进制形式，我将能够针对不同的训练集在超计划方程中绘制每个特征的权重分布.在这种情况下，我真的不需要PCA来查看超级计划的稳定性和功能的相对重要性(以中心为中心).在多类svm中，替代方案是什么，因为对于每个训练集，您有3个分类器，然后根据这3个分类器的结果选择一个班级(已经是什么?该班级出现的次数最多或判别较大) ?此处无关紧要的内容).任何人都有一个主意.

I'm classifing users with a multiclass svm (one-against-on), 3 classes. In binary, I would be able to plot the distribution of the weight of each feature in the hyperplan equation for different training sets. In this case, I don't really need a PCA to see stability of the hyperplan and relative importance of the features (reudced centered btw). What would the alternative be in multiclass svm, as for each training set you have 3 classifiers and you choose one class according to the result of the three classifiers (what is it already ? the class that appears the maximum number of times or the bigger discriminant ? whichever it does not really matter here). Anyone has an idea.

如果重要的话，我正在用Accord用C#编写. 谢谢！

And if it matters, I am writing in C# with Accord. Thank you !

推荐答案

在使用一对一策略的多类SVM中，该问题分为一组较小的二进制问题.例如，如果您有三个可能的类，则使用一对一策略需要创建(n(n-1))/n 二进制分类器.在您的示例中，这将是

In a multi-class SVM that uses the one-vs-one strategy, the problem is divided into a set of smaller binary problems. For example, if you have three possible classes, using the one-vs-one strategy requires the creation of (n(n-1))/n binary classifiers. In your example, this would be

(n(n-1))/n = (3(3-1))/2 = (3*2)/2 = 3

其中每个将专门解决以下问题:

Each of those will be specialized in the following problems:

• 区分第1类和第2类(我们将其称为 svm _a ).
• 区分第1类和第3类(我们称其为 svm _b )
• 区分第2类和第3类(我们称其为 svm _c )

现在，我看到您实际上在原始帖子中问了多个问题，因此我将分别询问.首先，我将阐明决策过程的工作原理，然后说明如何检测最重要的功能.

Now, I see that actually you have asked multiple questions in your original post, so I will ask them separately. First I will clarify how the decision process works, and then tell how you could detect which features are the most important.

自从您提到Accord.NET以来，该框架可能有两种方法来计算多类决策.默认情况是使用决策定向非循环图(DDAG)，仅不过是顺序消除类.另一种方法是解决所有二进制问题，并选择大多数时候都赢了的班.通过设置方法参数 SVM的计算方法.

Since you mentioned Accord.NET, there are two ways this framework might be computing the multi-class decision. The default one is to use a Decision Directed Acyclic Graph (DDAG), that is nothing more but the sequential elimination of classes. The other way is by solving all binary problems and taking the class that won most of the time. You can configure them at the moment you are classifying a new sample by setting the method parameter of the SVM's Compute method.

由于最容易上手的获胜版本易于理解，因此我将对默认方法DDAG进行更多说明.

Since the winning-most-of-the-time version is straightforward to understand, I will explain a little more about the default approach, the DDAG.

在此算法中，我们测试了每个SVM，并消除了每一轮丢失的类.因此，例如，该算法以所有可能的类开头:

In this algorithm, we test each of the SVMs and eliminate the class that lost at each round. So for example, the algorithm starts with all possible classes:

Candidate classes: [1, 2, 3]

现在，它要求 svm _a 进行x的分类，并确定类别2.因此，类别1丢失了，在以后的测试中不再考虑:

Now it asks svm_a to classify x, it decides for class 2. Therefore, class 1 lost and is not considered anymore in further tests:

Candidate classes: [2, 3]

现在，它要求 svm _b 对x进行分类，并确定为第2类.因此，第3类丢失了，在以后的测试中不再考虑:

Now it asks svm_b to classify x, it decides for class 2. Therefore, class 3 lost and is not considered anymore in further tests:

Candidate classes: [2]

因此，最终答案是2.

现在，由于一对一SVM被分解为(n(n-1)/2)个二进制问题，因此分析哪些特征最重要的最直接方法是分别考虑每个二进制问题.不幸的是，全局地确定哪个对整个问题最重要可能很棘手，但是可以区分出哪一个最重要来区分1级和2级，1级和3级，2级和3级.

Now, since the one-vs-one SVM is decomposed into (n(n-1)/2) binary problems, the most straightforward way to analyse which features are the most important is by considering each binary problem separately. Unfortunately it might be tricky to globally determine which are the most important for the entire problem, but it will be possible to detect which ones are the most important to discriminate between class 1 and 2, or class 1 and 3, or class 2 and 3.

但是，如果您使用的是DDAG，在这里我可以提供建议.使用DDAG，可以提取导致特定决策的决策路径 .这意味着可以对整个数据库进行分类时，估计每个二进制计算机使用了多少次.如果您可以估计每个二进制计算机功能的重要性，并估计数据库决策过程中使用一台计算机的次数，则可以将其加权总和作为功能在其中的有用性的指标.您的决策过程.

However, here I can offer a suggestion if your are using DDAGs. Using DDAGs, it is possible to extract the decision path that lead to a particular decision. This means that is it possible to estimate how many times each of the binary machines was used when classifying your entire database. If you can estimate the importance of a feature for each of the binary machines, and estimate how many times a machine is used during the decision process in your database, perhaps you could take their weighted sum as an indicator of how useful a feature is in your decision process.

顺便说一句，您可能也有兴趣尝试使用带有高C的L1正则化的Logistic回归支持向量机之一来执行稀疏特征选择:

By the way, you might also be interested in trying one of the Logistic Regression Support Vector Machines using L1-regularization with a high C to perform sparse feature selection:

// Create a new linear machine
var svm = new SupportVectorMachine(inputs: 2);

// Creates a new instance of the sparse logistic learning algorithm
var smo = new ProbabilisticCoordinateDescent(svm, inputs, outputs)
{
    // Set learning parameters
    Complexity = 100,
};

这篇关于在没有PCA的多类svm中找到正确的功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！