通过RandomizedSearchCV在Keras(MLP)中进行超参数调整 [英] Hyperparameter tuning in Keras (MLP) via RandomizedSearchCV

问题描述

一段时间以来，我一直在尝试调整神经网络，但不幸的是，我无法获得良好的性能.我有一个时间序列数据集，并且正在使用RandomizedSearchCV进行二进制分类.我的代码如下.任何建议或帮助，将不胜感激.一件事是，我仍在尝试找出合并的方法，那就是尽早停止.

I have been trying to tune a neural net for some time now but unfortunately, I cannot get a good performance out of it. I have a time-series dataset and I am using RandomizedSearchCV for binary classification. My code is below. Any suggestions or help will be appreciated. One thing is that I am still trying to figure out how to incorporate is early stopping.

忘了补充一点，我是基于F1宏度量标准来衡量性能的，我无法获得高于0.68的得分.我注意到的另一件事是，我尝试一次估计的参数越多(增加网格)，我的得分就越差.

Forgot to add that I am measuring the performance based on F1-macro metric and I cannot get a scoring higher that 0.68. Another thing that I noticed is that the more parameters I try to estimate at once (increase my grid), the worse my scoring is.

train_size = int(0.70*X.shape[0])
X_train, X_test, y_train, y_test = X[0:train_size], X[train_size:],y[0:train_size], y[train_size:]


from numpy.random import seed
seed(3)
from tensorflow import set_random_seed
set_random_seed(4)

from imblearn.pipeline import Pipeline

def create_model(activation_1='relu', activation_2='relu', 
                 neurons_input = 1, neurons_hidden_1=1,
                 optimizer='adam',
                 input_shape=(X_train.shape[1],)):

  model = Sequential()
  model.add(Dense(neurons_input, activation=activation_1, input_shape=input_shape, kernel_initializer='random_uniform'))

  model.add(Dense(neurons_hidden_1, activation=activation_2, kernel_initializer='random_uniform'))


  model.add(Dense(2, activation='sigmoid'))

  model.compile (loss = 'sparse_categorical_crossentropy', optimizer=optimizer)
  return model


clf=KerasClassifier(build_fn=create_model, verbose=0)

param_grid = {
    'clf__neurons_input':[5, 10, 15, 20, 25, 30, 35],
    'clf__neurons_hidden_1':[5, 10, 15, 20, 25, 30, 35],
    'clf__optimizer': ['Adam', 'Adamax','Adadelta'],
    'clf__activation_1': ['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear'],
    'clf__activation_2': ['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear'],
    'clf__batch_size': [40,60,80,100]}


pipe = Pipeline([
    ('oversample', SMOTE(random_state=12)),
    ('clf', clf)
    ])

my_cv = TimeSeriesSplit(n_splits=5).split(X_train)

rs_keras = RandomizedSearchCV(pipe, param_grid, cv=my_cv, scoring='f1_macro', refit='f1_macro', verbose=3, n_jobs=1,random_state=42)
rs_keras.fit(X_train, y_train)

print("Best: %f using %s" % (rs_keras.best_score_, rs_keras.best_params_))

from sklearn.metrics import classification_report, confusion_matrix
y_pred=rs_keras.predict(X_test)
clfreport = classification_report(y_test, y_pred)
cm = confusion_matrix(y_test, y_pred)
print (clfreport)
print (cm)
scores_test = rs_keras.score(X_test,y_test)
print ("Testing:", scores_test)

我的分数

推荐答案

关于EarlyStopping，

About EarlyStopping,

clf=KerasClassifier(build_fn=create_model, verbose=0)

stop = EarlyStopping(monitor='your_metric', min_delta=0, 
                         patience=5, verbose=1, mode='auto',
                         baseline=None, restore_best_weights=True)
.
.
.
grid.fit(x_train_sc, y_train_sc, callbacks = [stop])

应该可以. (我在没有管道结构的情况下进行了测试.)

It should work. (I tested it without pipeline structure.)

顺便说一句，当我尝试使用流水线结构的数据集时，它并没有达到我的预期.就我而言，我试图对数据进行StandardScale缩放，但是gridsearch并没有首先对数据进行缩放，因此它进入了分类器而不进行缩放.这对我来说是个问题.

By the way, when I was trying my dataset with pipeline structure, it did not act as I thought. In my case, I was trying to StandardScale the data but gridsearch did not scale the data first, so It went into the classifier without scaling. That was an issue for me.

我建议您在进行网格搜索之前先转换数据，然后尝试不使用管道.我知道数据泄漏问题，但找不到其他方法.

I suggest you the transform data before gridsearch and try without pipeline. I know about the data leakage problems but I couldn't find any other way.

这篇关于通过RandomizedSearchCV在Keras(MLP)中进行超参数调整的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！