动态时间规整内核返回错误率大于0的SVM [英] SVM with dynamic time warping kernel return error rate greater than 0
问题描述
我在研究中使用的是Accord.net. 我有一个可变大小的矢量序列作为输入，因此我将DynamicTimeWarping用作MulticlassSupportVectorMachine的内核.
I'm using Accord.net in my research. I have a vector sequences of variable size as input so I use DynamicTimeWarping as a kernel for MulticlassSupportVectorMachine.
IKernel kernel = new DynamicTimeWarping(dimension);
var machine = new MulticlassSupportVectorMachine(0, kernel, 2);
// Create the Multi-class learning algorithm for the machine
var teacher = new MulticlassSupportVectorLearning(machine, inputs.ToArray(), outputs.ToArray());
// Configure the learning algorithm to use SMO to train the
// underlying SVMs in each of the binary class subproblems.
teacher.Algorithm = (svm, classInputs, classOutputs, i, j) =>
new SequentialMinimalOptimization(svm, classInputs, classOutputs)
{
Complexity = 1.5
};
// Run the learning algorithm
double error = teacher.Run();
输入和输出如下:
?inputs.ToArray()
{double[22][]}
[0]: {double[10656]}
[1]: {double[9360]}
[2]: {double[9216]}
[3]: {double[9864]}
[4]: {double[10296]}
[5]: {double[10152]}
[6]: {double[9936]}
[7]: {double[9216]}
[8]: {double[10944]}
[9]: {double[9504]}
[10]: {double[11880]}
[11]: {double[22752]}
[12]: {double[23688]}
[13]: {double[29880]}
[14]: {double[32328]}
[15]: {double[37224]}
[16]: {double[30024]}
[17]: {double[27288]}
[18]: {double[26064]}
[19]: {double[22032]}
[20]: {double[21672]}
[21]: {double[22680]}
?inputs[0]
{double[10656]}
[0]: 7.6413027545068823
[1]: -61.607203372756942
[2]: 7.7375128997886513
[3]: -25.704529598536471
[4]: -0.4124927191531238
[5]: 9.6820255661415011
[6]: 3.0674374003781861
[7]: 4.6364653722537668
[8]: 3.3559314278499177
[9]: 0.93969394152714925
[10]: -6.3800159552064146
[11]: 1.4239779356781062
[12]: -2.25349154655782
[13]: -1.5457194406236221
[14]: -0.7612541874802764
[15]: -3.3364791133985348
[16]: 0.67816801816804861
[17]: -3.4117217877592343
[18]: 1.5785492543017225
[19]: 0.31091690789261689
[20]: -2.5526646739208712
[21]: -1.0550268575680164
[22]: -0.9598271201088191
[23]: -1.1797916101998056
[24]: 0.56157735657438412
[25]: -0.16309890421998655
[26]: 0.29765136770064271
[27]: -0.35684735108472643
[28]: -0.52382117896006564
[29]: -0.052087258844925849
[30]: -0.45363669419489172
[31]: -0.16216259086709361
[32]: -0.25958480481802632
[33]: 0.081248839173330589
[34]: -0.019783293216956807
[35]: 0.14139773316964666
[36]: 0.088466551256948273
[37]: -0.019528343614348152
[38]: 0.087073343332064762
[39]: 0.048432068369313144
[40]: -0.0069171431858626713
[41]: -0.0095272766950126042
[42]: 0.016639887499893875
[43]: -0.009108847017642599
[44]: 0.0017424263597747487
[45]: 0.0042160613810267641
[46]: -0.002793626734247919
[47]: 0.00092130299196750763
[48]: 0.0024488939699103319
[49]: 0.0021684669072286468
[50]: 0.000000000000010673294119695543
[51]: -0.000000000000014072530108313123
[52]: 0.000000000000000069063495074940116
[53]: 8.73342598612937E-17
[54]: 0.000000000000000030048643853749834
[55]: -6.95380121971215E-17
[56]: 0.00000000000000010093927767292201
[57]: 0.000000000000000046158366228268829
[58]: 0.000000000000000039070100378142324
[59]: 0.00000000000000010492059540665321
[60]: -0.000000000000000014254591247067773
[61]: -0.0000000000000000015902697756329909
[62]: 0.0000000000000000017024249964704589
[63]: 0.0000000000000000010277956708903136
[64]: 3.5875442986020568E-28
[65]: -2.215158998843094E-31
[66]: 1.041379591973569E-31
[67]: -4.3897715186113276E-31
[68]: 4.248432864156974E-34
[69]: 4.3718530099471368E-47
[70]: 1.4551856970655856E-50
[71]: 0.0
[72]: 11.031182384920639
[73]: -63.434486026723626
[74]: 1.7731679007864651
[75]: -23.968196466652273
[76]: 2.2753564408666507
[77]: 9.5492641110324534
[78]: 3.4465209481281054
[79]: 4.7979691924966161
[80]: 2.0147801482840508
[81]: 1.1858337013571998
[82]: -4.607944757859336
[83]: 0.75637871318664485
[84]: -3.8397810581420115
[85]: -2.1276086210477514
[86]: -0.4060620782117581
[87]: -2.9313848427777227
[88]: 0.052605148372525556
[89]: -1.5948208186863277
[90]: 0.36061926783486992
[91]: -0.12623742266247567
[92]: -1.1889713301479885
[93]: -0.33299631607409635
[94]: -0.00912650336180437
[95]: -0.52707950657313729
[96]: 0.52115933681848092
[97]: 0.46870463636533816
[98]: -0.18482093982467213
[99]: -0.49350561475314514
< More... (The first 100 of 10656 items were displayed.) >
?outputs
Count = 22
[0]: 0
[1]: 0
[2]: 0
[3]: 0
[4]: 0
[5]: 0
[6]: 0
[7]: 0
[8]: 0
[9]: 0
[10]: 0
[11]: 1
[12]: 1
[13]: 1
[14]: 1
[15]: 1
[16]: 1
[17]: 1
[18]: 1
[19]: 1
[20]: 1
[21]: 1
使用该代码，错误返回为0.5.
With that code, the error return is 0.5.
问题:
-
这是否意味着我的训练数据有问题?
Does that mean there are problems with my training data?
可变大小序列还可以使用其他内核吗?
Is there any other kernel that I can use with my variable size sequences?
谢谢.
推荐答案
我将举一个示例，说明如何使用DynamicTimeWarping内核与高斯内核相结合来执行序列分类，希望可以提供更好的结果.
I will give an example on how to perform sequence classification using the DynamicTimeWarping kernel combined with a Gaussian kernel, which should hopefully give better results.
序列分类问题中的第一个任务是正确组织序列以提供学习算法.每个序列都可以由多元向量组成，因此，输入数据必须进行相应的组织(这是使用序列机时最有可能产生错误的原因，因此请花一两分钟的时间来了解以下代码的作用).
The first task in a sequence classification problem is to organize the sequences properly to feed the learning algorithms. Each sequence can be composed of multivariate vectors, and as such, the input data must be organized accordingly (this is the most likely source of error when using the sequence machines, so please take a moment or two to understand what the following code is doing).
// Suppose you have sequences of multivariate observations, and that
// those sequences could be of arbitrary length. On the other hand,
// each observation have a fixed, delimited number of dimensions.
// In this example, we have sequences of 3-dimensional observations.
// Each sequence can have an arbitrary length, but each observation
// will always have length 3:
double[][][] sequences =
{
new double[][] // first sequence
{
new double[] { 1, 1, 1 }, // first observation of the first sequence
new double[] { 1, 2, 1 }, // second observation of the first sequence
new double[] { 1, 4, 2 }, // third observation of the first sequence
new double[] { 2, 2, 2 }, // fourth observation of the first sequence
},
new double[][] // second sequence (note that this sequence has a different length)
{
new double[] { 1, 1, 1 }, // first observation of the second sequence
new double[] { 1, 5, 6 }, // second observation of the second sequence
new double[] { 2, 7, 1 }, // third observation of the second sequence
},
new double[][] // third sequence
{
new double[] { 8, 2, 1 }, // first observation of the third sequence
},
new double[][] // fourth sequence
{
new double[] { 8, 2, 5 }, // first observation of the fourth sequence
new double[] { 1, 5, 4 }, // second observation of the fourth sequence
}
};
这些是我们的输入序列.现在，由于我们尝试执行分类问题，因此必须具有与每个序列相关联的输出类标签.如果我们有4个序列，那么我们将需要4个类标签:
Those are our input sequences. Now, since we are trying to perform a classification problem, we must have an output class label associated with each of those sequences. If we have 4 sequences, then we will need 4 class labels:
// Now, we will also have different class labels associated which each
// sequence. We will assign -1 to sequences whose observations start
// with { 1, 1, 1 } and +1 to those that do not:
int[] outputs =
{
-1,-1, // First two sequences are of class -1 (those start with {1,1,1})
1, 1, // Last two sequences are of class +1 (don't start with {1,1,1})
};
现在已经定义了问题，必须对其进行一些转换，以便DTW-SVM可以对其进行处理:
Now that the problem has been defined, it has to be transformed a little so they can be processed by the DTW-SVMs:
// At this point, we will have to "flat" out the input sequences from double[][][]
// to a double[][] so they can be properly understood by the SVMs. The problem is
// that, normally, SVMs usually expect the data to be comprised of fixed-length
// input vectors and associated class labels. But in this case, we will be feeding
// them arbitrary-length sequences of input vectors and class labels associated with
// each sequence, instead of each vector.
double[][] inputs = new double[sequences.Length][];
for (int i = 0; i < sequences.Length; i++)
inputs[i] = Matrix.Concatenate(sequences[i]);
// Now we have to setup the Dynamic Time Warping kernel. We will have to
// inform the length of the fixed-length observations contained in each
// arbitrary-length sequence:
//
var kernel = new Gaussian<DynamicTimeWarping>(new DynamicTimeWarping(length: 3));
// Now we can create the machine. When using variable-length
// kernels, we will need to pass zero as the input length:
var svm = new KernelSupportVectorMachine(kernel, inputs: 0);
// Create the Sequential Minimal Optimization learning algorithm
var smo = new SequentialMinimalOptimization(svm, inputs, outputs);
// And start learning it!
double error = smo.Run(); // error will be 0.0
此后，应该已经创建了机器.机器应该已经自动猜测了参数C，但是您可以在之后对其进行微调，以尝试提高机器的泛化性能.无论如何，
After this point, the machine should have been created. The parameter C should have been guessed automatically by the machine, but you can fine tune it afterwards to try to increase the generalization performance of your machines. In any case,
// At this point, we should have obtained an useful machine. Let's
// see if it can understand a few examples it hasn't seem before:
double[][] a =
{
new double[] { 1, 1, 1 },
new double[] { 7, 2, 5 },
new double[] { 2, 5, 1 },
};
double[][] b =
{
new double[] { 7, 5, 2 },
new double[] { 4, 2, 5 },
new double[] { 1, 1, 1 },
};
// Following the aforementioned logic, sequence (a) should be
// classified as -1, and sequence (b) should be classified as +1.
int resultA = System.Math.Sign(svm.Compute(Matrix.Concatenate(a))); // -1
int resultB = System.Math.Sign(svm.Compute(Matrix.Concatenate(b))); // +1
这篇关于动态时间规整内核返回错误率大于0的SVM的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
