插入符号中的用户定义的 summaryFunction,logloss [英] user defined summaryFunction in caret, logloss
问题描述
使用 caret 包时,我无法使以下用户定义的汇总函数正常工作.它应该计算 logloss,但我一直发现找不到 logloss.下面是一个可重现的示例:
Using the caret package, I am having trouble getting the following user defined summary function to work. It is supposed to calculate the logloss, but I keep getting that logloss is not found. Below, a reproducible example:
data <- data.frame('target' = sample(c('Y','N'),100,replace = T), 'X1' = runif(100), 'X2' = runif(100))
log.loss2 <- function(data, lev = NULL, model = NULL) {
logloss = -sum(data$obs*log(data$Y) + (1-data$obs)*log(1-data$Y))/length(data$obs)
names(logloss) <- c('LL')
logloss
}
fitControl <- trainControl(method="cv",number=1, classProbs = T, summaryFunction = log.loss2)
my.grid <- expand.grid(.decay = c(0.05), .size = c(2))
fit.nnet2 <- train(target ~., data = data,
method = "nnet", maxit = 500, metric = 'LL',
tuneGrid = my.grid, verbose = T)
推荐答案
错误是由于您没有在调用 train 时包含 trControl = fitControl
.然而,这会给你带来另一个错误,这是由于 data$obs
和 data$pred
是因素 - 需要转换为数字,这给 1
或 2
,减去 1
得到所需的 0
和 1
The error was due to the fact you did not include trControl = fitControl
in the call to train. However that would bring you to another error that is due to the fact data$obs
and data$pred
are factors - one needs to convert to numeric which gives 1
or 2
, subtracting 1
gives desired 0
and 1
log.loss2 <- function(data, lev = NULL, model = NULL) {
data$pred <- as.numeric(data$pred)-1
data$obs <- as.numeric(data$obs)-1
logloss = -sum(data$obs*log(data$Y) + (1-data$obs)*log(1-data$Y))/length(data$obs)
names(logloss) <- c('LL')
logloss
}
fitControl <- trainControl(method="cv",number=1, classProbs = T, summaryFunction = log.loss2)
fit.nnet2 <- train(target ~., data = data,
method = "nnet", maxit = 500, metric = "LL" ,
tuneGrid = my.grid, verbose = T, trControl = fitControl,
maximize = FALSE)
#output
Neural Network
100 samples
2 predictor
2 classes: 'N', 'Y'
No pre-processing
Resampling: Cross-Validated (1 fold)
Summary of sample sizes: 0
Resampling results:
LL
0.6931472
Tuning parameter 'size' was held constant at a value of 2
Tuning parameter 'decay' was held constant at a value of 0.05
需要注意的几点:
此损失函数仅适用于包含 N
/Y
作为类的数据,因为概率定义为 data$Y
,这是一种更好的方法是找到类的名称并使用它.此外,截断概率值的良好做法是因为 log(0)
不是一个好主意:
this loss function will work only with data containing N
/Y
as classes because probability is defined as data$Y
, a better approach is to find the name of the class and use that. Additionally its good practice to truncate the probability values since log(0)
is not a good idea:
LogLoss <- function (data, lev = NULL, model = NULL)
{
obs <- data[, "obs"]
cls <- levels(obs) #find class names
probs <- data[, cls[2]] #use second class name
probs <- pmax(pmin(as.numeric(probs), 1 - 1e-15), 1e-15) #bound probability
logPreds <- log(probs)
log1Preds <- log(1 - probs)
real <- (as.numeric(data$obs) - 1)
out <- c(mean(real * logPreds + (1 - real) * log1Preds)) * -1
names(out) <- c("LogLoss")
out
}
这篇关于插入符号中的用户定义的 summaryFunction,logloss的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!