为什么我在使用 Keras 与随机森林或 knn 时得到了糟糕的结果? [英] Why I'm getting bad result with Keras vs random forest or knn?

问题描述

我正在使用 keras 学习深度学习，并尝试将结果(准确性)与机器学习算法 (sklearn)(即随机森林)进行比较code>, k_neighbors)

似乎使用 keras 我得到了最糟糕的结果.我正在处理简单的分类问题:iris dataset我的 keras 代码看起来:

samples = datasets.load_iris()X = 样本.数据y = 样本.目标df = pd.DataFrame(data=X)df.columns = samples.feature_namesdf['目标'] = y# 准备数据X = df[df.columns[:-1]]y = df[df.columns[-1]]# 热编码编码器 = LabelEncoder()y1 = 编码器.fit_transform(y)y = pd.get_dummies(y1).values# 分裂X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3)# 构建模型模型 = 顺序()model.add(Dense(1000, activation='tanh', input_shape = ((df.shape[1]-1),)))model.add(Dense(500, activation='tanh'))model.add(Dense(250, activation='tanh'))model.add(Dense(125, activation='tanh'))model.add(Dense(64, activation='tanh'))model.add(Dense(32, activation='tanh'))模型.添加(密集(9，激活=tanh"))model.add(Dense(y.shape[1], activation='softmax'))模型编译(优化器='亚当'，损失='categorical_crossentropy'，指标=['准确度'])模型.fit(X_train，y_train)分数，acc = model.evaluate(X_test, y_test,verbose=0)#结果:#score = 0.77#acc = 0.711

我厌倦了添加层和/或更改每层的单元数和/或更改激活函数(到 relu)，结果似乎不高于 0.85.

使用 sklearn random forest 或 k_neighbors 我得到的结果(在同一数据集上)高于 0.95.

1. 我错过了什么?

2. 使用 sklearn 我做了很少的努力并得到了不错的结果，使用 keras，我有很多升级但不如 sklearn 结果.为什么会这样?

3. 如何使用 keras 获得相同的结果?

解决方案

简而言之，您需要:

1. ReLU 激活
2. 更简单的模型
3. 数据规范化
4. 更多时代

详细说明:

这里的第一个问题是，如今我们从不将 activation='tanh' 用于中间网络层.在此类问题中，我们实际上总是使用 activation='relu'.

第二个问题是你已经构建了一个相当大的 Keras 模型，很可能你的训练集中只有 100 个虹膜样本，你的数据太少，无法有效地训练这么大的模型.尝试大幅减少层数和每层节点数.开始更简单.

当我们拥有大量数据时，大型神经网络确实会蓬勃发展，但在像这里这样的小型数据集的情况下，与更简单的算法(如 RF 或k-nn.

第三个问题是，与基于树的模型(如随机森林)相比，神经网络通常需要对数据进行归一化，而您不需要这样做.事实是 knn 也需要归一化数据，但在这种特殊情况下，由于所有虹膜特征都在同一尺度上，因此不会对性能产生负面影响.

最后但并非最不重要的是，您似乎只在一个时期内运行您的 Keras 模型(如果您未在 model.fit 中指定任何内容，则为默认值)；这有点等同于用一棵树构建一个随机森林(顺便说一句，它仍然是 比单个决策树好得多).

总而言之，在您的代码中进行以下更改:

from sklearn.preprocessing import StandardScalersc = StandardScaler()X_train = sc.fit_transform(X_train)X_test = sc.transform(X_test)模型 = 顺序()model.add(Dense(150, activation='relu', input_shape = ((df.shape[1]-1),)))model.add(Dense(150, activation='relu'))model.add(Dense(y.shape[1], activation='softmax'))模型.fit(X_train, y_train, epochs=100)

以及其他所有内容，我们得到:

score, acc = model.evaluate(X_test, y_test,verbose=0)ACC# 0.9333333373069763

我们可以做得更好:使用稍微更多的训练数据并将它们分层，即

X_train, X_test, y_train, y_test = train_test_split(X, y,test_size = 0.20, # 多几个训练样本分层=y)

并与相同的模型 &您可以在测试集中获得 1.0 的完美准确率:

score, acc = model.evaluate(X_test, y_test,verbose=0)ACC# 1.0

(由于此类实验中默认强加的一些随机性，细节可能有所不同).

I'm learning deep learning with keras and trying to compare the results (accuracy) with machine learning algorithms (sklearn) (i.e random forest, k_neighbors)

It seems that with keras I'm getting the worst results. I'm working on simple classification problem: iris dataset My keras code looks:

samples = datasets.load_iris()
X = samples.data
y = samples.target
df = pd.DataFrame(data=X)
df.columns = samples.feature_names
df['Target'] = y

# prepare data
X = df[df.columns[:-1]]
y = df[df.columns[-1]]

# hot encoding
encoder = LabelEncoder()
y1 = encoder.fit_transform(y)
y = pd.get_dummies(y1).values

# split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3)

# build model
model = Sequential()
model.add(Dense(1000, activation='tanh', input_shape = ((df.shape[1]-1),)))
model.add(Dense(500, activation='tanh'))
model.add(Dense(250, activation='tanh'))
model.add(Dense(125, activation='tanh'))
model.add(Dense(64, activation='tanh'))
model.add(Dense(32, activation='tanh'))
model.add(Dense(9, activation='tanh'))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train)
score, acc = model.evaluate(X_test, y_test, verbose=0)

#results:
#score = 0.77
#acc = 0.711

I have tired to add layers and/or change number of units per layer and/or change the activation function (to relu) by it seems that the result are not higher than 0.85.

With sklearn random forest or k_neighbors I'm getting result (on same dataset) above 0.95.

1. What am I missing ?

2. With sklearn I did little effort and got good results, and with keras, I had a lot of upgrades but not as good as sklearn results. why is that ?

3. How can I get same results with keras ?

解决方案

In short, you need:

1. ReLU activations
2. Simpler model
3. Data mormalization
4. More epochs

In detail:

The first issue here is that nowadays we never use activation='tanh' for the intermediate network layers. In such problems, we practically always use activation='relu'.

The second issue is that you have build quite a large Keras model, and it might very well be the case that with only 100 iris samples in your training set you have too few data to effectively train such a large model. Try reducing drastically both the number of layers and the number of nodes per layer. Start simpler.

Large neural networks really thrive when we have lots of data, but in cases of small datasets, like here, their expressiveness and flexibility may become a liability instead, compared with simpler algorithms, like RF or k-nn.

The third issue is that, in contrast to tree-based models, like Random Forests, neural networks generally require normalizing the data, which you don't do. Truth is that knn also requires normalized data, but in this special case, since all iris features are in the same scale, it does not affect the performance negatively.

Last but not least, you seem to run your Keras model for only one epoch (the default value if you don't specify anything in model.fit); this is somewhat equivalent to building a random forest with a single tree (which, BTW, is still much better than a single decision tree).

All in all, with the following changes in your code:

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

model = Sequential()
model.add(Dense(150, activation='relu', input_shape = ((df.shape[1]-1),)))
model.add(Dense(150, activation='relu'))
model.add(Dense(y.shape[1], activation='softmax'))

model.fit(X_train, y_train, epochs=100)

and everything else as is, we get:

score, acc = model.evaluate(X_test, y_test, verbose=0)
acc
# 0.9333333373069763

We can do better: use slightly more training data and stratify them, i.e.

X_train, X_test, y_train, y_test = train_test_split(X, y, 
                                                    test_size = 0.20, # a few more samples for training
                                                    stratify=y)

And with the same model & training epochs you can get a perfect accuracy of 1.0 in the test set:

score, acc = model.evaluate(X_test, y_test, verbose=0)
acc
# 1.0

(Details might differ due to some randomness imposed by default in such experiments).

这篇关于为什么我在使用 Keras 与随机森林或 knn 时得到了糟糕的结果?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！