R中的手动Perceptron示例-结果可接受吗? [英] Manual Perceptron example in R - are the results acceptable?

问题描述

我正在尝试使用感知器算法进行分类工作，但我认为缺少某些东西.这是通过逻辑回归实现的决策边界:

在测试1和2表现更好之后，红点进入了大学.

与逻辑回归的训练分类性能一致，该代码在概念上可能不是错误的.

问题:获得与逻辑回归如此不同的系数是否可以:

 (拦截)test1 test21.718449 4.012903 3.743903 

解决方案

这实际上是一个CrossValidated问题，而不是StackOverflow问题，但我会继续回答.

是的，由于您无法直接比较这两种技术之间的系数幅度，因此正常情况下并期望得到非常不同的系数.

对于logit(物流)模型，您使用的是二项式分布和基于S形成本函数的logit-link.系数仅在这种情况下才有意义.您在Logit中也有一个拦截项.

对于感知器模型，这都不是正确的.因此，系数的解释完全不同.

现在，这并不是说哪种型号更好.您的问题中没有可比的绩效指标，无法让我们确定.要确定您应该进行交叉验证还是至少使用保留样本.

I am trying to get a perceptron algorithm for classification working but I think something is missing. This is the decision boundary achieved with logistic regression:

The red dots got into college, after performing better on tests 1 and 2.

This is the data, and this is the code for the logistic regression in R:

dat = read.csv("perceptron.txt", header=F)
colnames(dat) = c("test1","test2","y")
plot(test2 ~ test1, col = as.factor(y), pch = 20, data=dat)
fit = glm(y ~ test1 + test2, family = "binomial", data = dat)
coefs = coef(fit)
(x = c(min(dat[,1])-2,  max(dat[,1])+2))
(y = c((-1/coefs[3]) * (coefs[2] * x + coefs[1])))
lines(x, y)

The code for the "manual" implementation of the perceptron is as follows:

# DATA PRE-PROCESSING:
dat = read.csv("perceptron.txt", header=F)
dat[,1:2] = apply(dat[,1:2], MARGIN = 2, FUN = function(x) scale(x)) # scaling the data
data = data.frame(rep(1,nrow(dat)), dat) # introducing the "bias" column
colnames(data) = c("bias","test1","test2","y")
data$y[data$y==0] = -1 # Turning 0/1 dependent variable into -1/1.
data = as.matrix(data) # Turning data.frame into matrix to avoid mmult problems.

# PERCEPTRON:
set.seed(62416)
no.iter = 1000                           # Number of loops
theta = rnorm(ncol(data) - 1)            # Starting a random vector of coefficients.
theta = theta/sqrt(sum(theta^2))         # Normalizing the vector.
h = theta %*% t(data[,1:3])              # Performing the first f(theta^T X)

for (i in 1:no.iter){                    # We will recalculate 1,000 times
  for (j in 1:nrow(data)){               # Each time we go through each example.
      if(h[j] * data[j, 4] < 0){         # If the hypothesis disagrees with the sign of y,
      theta = theta + (sign(data[j,4]) * data[j, 1:3]) # We + or - the example from theta.
      }
      else
      theta = theta                      # Else we let it be.
  }
  h = theta %*% t(data[,1:3])            # Calculating h() after iteration.
}
theta                                    # Final coefficients
mean(sign(h) == data[,4])                # Accuracy

With this, I get the following coefficients:

     bias     test1     test2 
 9.131054 19.095881 20.736352 

and an accuracy of 88%, consistent with that calculated with the glm() logistic regression function: mean(sign(predict(fit))==data[,4]) of 89% - logically, there is no way of linearly classifying all of the points, as it is obvious from the plot above. In fact, iterating only 10 times and plotting the accuracy, a ~90% is reach after just 1 iteration:

Being in line with the training classification performance of logistic regression, it is likely that the code is not conceptually wrong.

QUESTIONS: Is it OK to get coefficients so different from the logistic regression:

(Intercept)       test1       test2 
   1.718449    4.012903    3.743903 

解决方案

This is really more of a CrossValidated question than a StackOverflow question, but I'll go ahead and answer.

Yes, it's normal and expected to get very different coefficients because you can't directly compare the magnitude of the coefficients between these 2 techniques.

With the logit (logistic) model you're using a binomial distribution and logit-link based on a sigmoid cost function. The coefficients are only meaningful in this context. You've also got an intercept term in the logit.

None of this is true for the perceptron model. The interpretation of the coefficients are thus totally different.

Now, that's not saying anything about which model is better. There aren't comparable performance metrics in your question that would allow us to determine that. To determine that you should do cross-validation or at least use a holdout sample.

这篇关于R中的手动Perceptron示例-结果可接受吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！