手动计算支持向量机的决策函数 [英] Calculating decision function of SVM manually
问题描述
我正在尝试使用python库SKLearn手动(与使用内置方法相对)计算SVC分类器的Decision_function.
I'm attempting to calculate the decision_function of a SVC classifier MANUALLY (as opposed to using the inbuilt method) using the the python library SKLearn.
我尝试了几种方法,但是,只有在不缩放数据时,我才能获得与之匹配的手动计算.
I've tried several methods, however, I can only ever get the manual calculation to match when I don't scale my data.
z
是一个测试数据(已按比例缩放),我认为其他变量可以说明一切(此外,如果代码中不明显,我将使用rbf内核).
z
is a test datum (that's been scaled) and I think the other variables speak for themselves (also, I'm using an rbf kernel if thats not obvious from the code).
以下是我尝试过的方法:
Here are the methods that I've tried:
dec_func = 0
for j in range(np.shape(sup_vecs)[0]):
norm2 = np.linalg.norm(sup_vecs[j, :] - z)**2
dec_func = dec_func + dual_coefs[0, j] * np.exp(-gamma*norm2)
dec_func += intercept
2矢量化方法
diff = sup_vecs - z
norm2 = np.sum(np.sqrt(diff*diff), 1)**2
dec_func = dual_coefs.dot(np.exp(-gamma_params*norm2)) + intercept
但是,它们都不返回与decision_function
相同的值.我认为这可能与重新调整我的价值观有关,或者更有可能是我一直在寻找的愚蠢事物!
However, neither of these ever returns the same value as decision_function
. I think it may have something to do with rescaling my values or more likely its something silly that I've been over looking!
任何帮助将不胜感激.
推荐答案
因此,在进行更多挖掘和抓挠之后,我已经弄清楚了.
So after a bit more digging and head scratching, I've figured it out.
如上所述,z
是已缩放的测试数据.要进行缩放,我必须从preprocessing.StandardScaler()对象中提取.mean_
和.std_
属性(当然是在训练数据上调用.fit()之后).
As I mentioned above z
is a test datum that's been scaled. To scale it I had to extract .mean_
and .std_
attributes from the preprocessing.StandardScaler() object (after calling .fit() on my training data of course).
然后我将缩放后的z
用作我的手动计算和内置函数的输入.但是,内置函数是管道的一部分,该管道已经将StandardScaler作为其管道中的第一个管道",结果z
被缩放了两次!
因此,当我从管道中删除缩放比例时,手册回答匹配"了内置函数的回答.
I was then using this scaled z
as an input to both my manual calculations and to the inbuilt function. However the inbuilt function was a part of a pipeline which already had StandardScaler as its first 'pipe' in the pipeline and as a result z
was getting scaled twice!
Hence, when I removed scaling from my pipeline, the manual answers "matched" the inbuilt function's answer.
我用引号说匹配",因为我总是不得不翻转手动计算的符号以匹配内置版本.目前我不知道为什么会这样.
I say "matched" in quotes by the way as I found I always had to flip the sign of my manual calculations to match the inbuilt version. Currently I have no idea why this is the case.
最后,我误解了管道的工作原理.
To conclude, I misunderstood how pipelines worked.
对于那些感兴趣的人,这是我的手动方法的最终版本:
For those that are interested, here's the final versions of my manual methods:
diff = sup_vecs - z_scaled
# Looping Method
dec_func_loop = 0
for j in range(np.shape(sup_vecs)[0]):
norm2 = np.linalg.norm(diff[j,:])
dec_func_loop = dec_func_loop + dual_coefs[j] * np.exp(-gamma*(norm2**2))
dec_func_loop = -1 * (dec_func_loop - intercept)
# Vectorized method
norm2 = np.array([np.linalg.norm(diff[n, :]) for n in range(np.shape(sup_vecs)[0])])
dec_func_vec = -1 * (dual_coefs.dot(np.exp(-gamma*(norm2**2))) - intercept)
附录
对于那些对实现多类SVC的手动方法感兴趣的人,以下链接会有所帮助: https://stackoverflow. com/a/27752709/1182556
这篇关于手动计算支持向量机的决策函数的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!