keras/scikit-learn:使用带有交叉验证的fit_generator() [英] keras/scikit-learn: using fit_generator() with cross validation

问题描述

是否可以将 Keras的scikit-learn API 与fit_generator()方法一起使用?还是使用另一种方式来批量生产训练?我正在使用SciPy的稀疏矩阵，在将其输入Keras之前必须将其转换为NumPy数组，但是由于内存消耗高，我无法同时转换它们.这是我的批量生产功能:

Is it possible to use Keras's scikit-learn API together with fit_generator() method? Or use another way to yield batches for training? I'm using SciPy's sparse matrices which must be converted to NumPy arrays before input to Keras, but I can't convert them at the same time because of high memory consumption. Here is my function to yield batches:

def batch_generator(X, y, batch_size):
    n_splits = len(X) // (batch_size - 1)
    X = np.array_split(X, n_splits)
    y = np.array_split(y, n_splits)

    while True:
        for i in range(len(X)):
            X_batch = []
            y_batch = []
            for ii in range(len(X[i])):
                X_batch.append(X[i][ii].toarray().astype(np.int8)) # conversion sparse matrix -> np.array
                y_batch.append(y[i][ii])
            yield (np.array(X_batch), np.array(y_batch))

和带有交叉验证的示例代码:

and example code with cross validation:

from sklearn.model_selection import StratifiedKFold, GridSearchCV
from sklearn import datasets

from keras.models import Sequential
from keras.layers import Activation, Dense
from keras.wrappers.scikit_learn import KerasClassifier

import numpy as np


def build_model(n_hidden=32):
    model = Sequential([
        Dense(n_hidden, input_dim=4),
        Activation("relu"),
        Dense(n_hidden),
        Activation("relu"),
        Dense(3),
        Activation("sigmoid")
    ])
    model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
    return model


iris = datasets.load_iris()
X = iris["data"]
y = iris["target"].flatten()

param_grid = {
    "n_hidden": np.array([4, 8, 16]),
    "nb_epoch": np.array(range(50, 61, 5))
}

model = KerasClassifier(build_fn=build_model, verbose=0)
skf = StratifiedKFold(n_splits=5).split(X, y) # this yields (train_indices, test_indices)

grid = GridSearchCV(model, param_grid, cv=skf, verbose=2, n_jobs=4)
grid.fit(X, y)

print(grid.best_score_)
print(grid.cv_results_["params"][grid.best_index_])

为进一步解释，它使用param_grid中所有可能的超参数组合来构建模型.然后在StratifiedKFold提供的训练数据分割(折叠)上逐一训练和测试每个模型.那么，给定模型的最终得分是各方面的平均得分.

To explain it more, it uses all the possible combinations of hyper-parameters in param_grid to build a model. Each model is then trained and tested one by one on the train-test data splits (folds) provided by StratifiedKFold. Then final score for a given model is a mean score from all folds.

那么是否可以在上面的代码中插入一些预处理子步骤以在实际拟合之前转换数据(稀疏矩阵)?

我知道我可以编写自己的交叉验证生成器，但是它必须产生索引，而不是真实数据！

I know I can write my own cross validation generator, but it must yield indexes, not the real data!

推荐答案

实际上，您可以使用生成器将稀疏矩阵用作Keras的输入.这是我在先前项目中使用的版本:

Actually you can use a sparse matrix as input to Keras with a generator. Here is my version that worked on a previous project:

> class KerasClassifier(KerasClassifier):
>     """ adds sparse matrix handling using batch generator
>     """
>     
>     def fit(self, x, y, **kwargs):
>         """ adds sparse matrix handling """
>         if not issparse(x):
>             return super().fit(x, y, **kwargs)
>         
>         ############ adapted from KerasClassifier.fit   ######################   
>         if self.build_fn is None:
>             self.model = self.__call__(**self.filter_sk_params(self.__call__))
>         elif not isinstance(self.build_fn, types.FunctionType):
>             self.model = self.build_fn(
>                 **self.filter_sk_params(self.build_fn.__call__))
>         else:
>             self.model = self.build_fn(**self.filter_sk_params(self.build_fn))
> 
>         loss_name = self.model.loss
>         if hasattr(loss_name, '__name__'):
>             loss_name = loss_name.__name__
>         if loss_name == 'categorical_crossentropy' and len(y.shape) != 2:
>             y = to_categorical(y)
>         ### fit => fit_generator
>         fit_args = copy.deepcopy(self.filter_sk_params(Sequential.fit_generator))
>         fit_args.update(kwargs)
>         ############################################################
>         self.model.fit_generator(
>                     self.get_batch(x, y, self.sk_params["batch_size"]),
>                                         samples_per_epoch=x.shape[0],
>                                         **fit_args)                      
>         return self                               
> 
>     def get_batch(self, x, y=None, batch_size=32):
>         """ batch generator to enable sparse input """
>         index = np.arange(x.shape[0])
>         start = 0
>         while True:
>             if start == 0 and y is not None:
>                 np.random.shuffle(index)
>             batch = index[start:start+batch_size]
>             if y is not None:
>                 yield x[batch].toarray(), y[batch]
>             else:
>                 yield x[batch].toarray()
>             start += batch_size
>             if start >= x.shape[0]:
>                 start = 0
>   
>     def predict_proba(self, x):
>         """ adds sparse matrix handling """
>         if not issparse(x):
>             return super().predict_proba(x)
>             
>         preds = self.model.predict_generator(
>                     self.get_batch(x, None, self.sk_params["batch_size"]), 
>                                                val_samples=x.shape[0])
>         return preds

这篇关于keras/scikit-learn:使用带有交叉验证的fit_generator()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！