为类概率稳定神经网络预测 [英] Stabilize Neural network prediction for class probability

问题描述

我一直在尝试使用library(keras)将神经网络用于二进制设置，并且我对类概率(而不是事件的概率0/1)感兴趣

I ve been trying to fit a neural network for binary setting using library(keras) and I am interested in class probability (instead of 0/1, probability of the event)

我的负面评价是正面评价的5.018倍.我添加了我一直在使用的代码.我无法稳定这些预测.我明白那杂音和一切. 但是我需要设置一些约束来每次获得接近的估计. 我没有想法.还有什么我可以用来稳定预测的东西吗?

I ve 5.018 times more negative than positive class. I added the code I have been using. I cannot stabilize the predictions. I understand that noise and everything. But I need to put some constraints to get close estimates each time. I am out of ides. Is there anything else I can use to stabilize predictions?

我无法共享数据，因此这里是火车数据级别的预测摘要，并绘制了验证/火车图.

I cannot share the data therefore here is summary of predictions at train data level and I plotted validations/train.

 first run               Second run
 Min.   :0.001843       Min.   :0.0004508 
 1st Qu.:0.012272       1st Qu.:0.0156236 
 Median :0.042264       Median :0.0459510 
 Mean   :0.142551       Mean   :0.1400624  
 3rd Qu.:0.195536       3rd Qu.:0.1937293
 Max.   :0.919892       Max.   :0.9882065 

首次运行的验证图和第二次运行的验证图

validation plot for first run and validation plot for second run

l2_model <- 
  keras_model_sequential() %>%
  layer_dense(units = 512, activation = "relu", input_shape =  ncol(XX_train1),
              kernel_regularizer = regularizer_l2(0.001)) %>% 
  layer_batch_normalization()%>%
  layer_dense(units = 256, activation = "relu", 
              kernel_regularizer = regularizer_l2(0.001)) %>%
  layer_batch_normalization()%>%
  layer_dense(units = 1, activation = "sigmoid",
              bias_initializer = initializer_constant(log(5.0189)))

l2_model %>% compile(
  optimizer="Adam",
  loss = "binary_crossentropy",
  metrics =  c('accuracy')
)

summary(l2_model)

l2_history <- l2_model %>% fit(
  x                = as.matrix(XX_train1), 
  y                = YY_train1,
  epochs = 30,
  batch_size = 1000,
  validation_data = list(XX_test, YY_test[,2]),
  verbose = 2,
  callbacks = list(
    callback_early_stopping(patience = 2) )
 #   ,callback_reduce_lr_on_plateau()  )
)


# Predicted Class Probability
yhat_keras_prob_vec  <- predict_proba(object = l2_model, x = as.matrix(XX_train1)) %>%
  as.matrix()

summary(yhat_keras_prob_vec)

推荐答案

所以我一直在努力，我开始控制一些东西，以获得诸如learning rate和decay之类的代码的近似估计.此optimizer=optimizer_adam(lr = 0.0001,decay = 0.001)，然后在每个layer_dense()并最终输出层中，将所有正则化器kernel_regularizer，bias_regularizer和activity_regularizer用作 l2正则化器，我仅使用 bias和活动正则化器

So I ve been working on and I started controlling bunch on stuff to get kind of close estimates such as learning rate and decay part of code is like this optimizer=optimizer_adam(lr = 0.0001,decay = 0.001) then I used all regularizers kernel_regularizer, bias_regularizer and activity_regularizer as l2 regularizer in each layer_dense() and finally output layer, I only used bias and activity regularizer.

这篇关于为类概率稳定神经网络预测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！