多标签二进制分类中的Keras class_weight [英] Keras class_weight in multi-label binary classification
问题描述
使用class_weight解决多标签问题.也就是说,每个标签都是0或1,但是每个输入样本都有很多标签.
Having trouble using class_weight for my multi-label problem. That is, each label is either 0 or 1, but there are many labels for each input sample.
代码(用于MWE的随机数据):
The code (with random data for MWE purposes):
import tensorflow as tf
from keras.models import Sequential, Model
from keras.layers import Input, Concatenate, LSTM, Dense
from keras import optimizers
from keras.utils import to_categorical
from keras import backend as K
import numpy as np
# from http://www.deepideas.net/unbalanced-classes-machine-learning/
def sensitivity(y_true, y_pred):
true_positives = tf.reduce_sum(tf.round(K.clip(y_true * y_pred, 0, 1)))
possible_positives = tf.reduce_sum(tf.round(K.clip(y_true, 0, 1)))
return true_positives / (possible_positives + K.epsilon())
# from http://www.deepideas.net/unbalanced-classes-machine-learning/
def specificity(y_true, y_pred):
true_negatives = tf.reduce_sum(K.round(K.clip((1-y_true) * (1-y_pred), 0, 1)))
possible_negatives = tf.reduce_sum(K.round(K.clip(1-y_true, 0, 1)))
return true_negatives / (possible_negatives + K.epsilon())
def to_train(a_train, y_train):
hours_np = [np.arange(a_train.shape[1])]*a_train.shape[0]
train_hours = to_categorical(hours_np)
n_samples = a_train.shape[0]
n_classes = 4
features_in = np.zeros((n_samples, n_classes))
supp_feat = np.random.choice(n_classes, n_samples)
features_in[np.arange(n_samples), supp_feat] = 1
#This model has 3 separate inputs
seq_model_in = Input(shape=(1,),batch_shape=(1, 1, a_train.shape[2]), name='seq_model_in')
feat_in = Input(shape=(1,), batch_shape=(1, features_in.shape[1]), name='feat_in')
feat_dense = Dense(1)(feat_in)
hours_in = Input(shape=(1,), batch_shape=(1, 1, train_hours.shape[2]), name='hours_in')
#Model intermediate layers
t_concat = Concatenate(axis=-1)([seq_model_in, hours_in])
lstm_layer = LSTM(1, batch_input_shape=(1, 1, (a_train.shape[2]+train_hours.shape[2])), return_sequences=False, stateful=True)(t_concat)
merged_after_lstm = Concatenate(axis=-1)([lstm_layer, feat_dense]) #may need another Dense() after
dense_merged = Dense(a_train.shape[2], activation="sigmoid")(merged_after_lstm)
#Define input and output to create model, and compile
model = Model(inputs=[seq_model_in, feat_in, hours_in], outputs=dense_merged)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=[sensitivity, specificity])
class_weights = {0.:1., 1.:118.}
seq_length = 23
#TRAINING (based on http://philipperemy.github.io/keras-stateful-lstm/)
for epoch in range(2):
for i in range(a_train.shape[0]):
y_true_1 = np.expand_dims(y_train[i,:], axis=1)
y_true = np.swapaxes(y_true_1, 0, 1)
#print 'y_true', y_true.shape
for j in range(seq_length-1):
input_1 = np.expand_dims(np.expand_dims(a_train[i][j], axis=1), axis=1)
input_1 = np.reshape(input_1, (1, 1, a_train.shape[2]))
input_2 = np.expand_dims(np.array(features_in[i]), axis=1)
input_2 = np.swapaxes(input_2, 0, 1)
input_3 = np.expand_dims(np.array([train_hours[i][j]]), axis=1)
tr_loss, tr_sens, tr_spec = model.train_on_batch([input_1, input_2, input_3], y_true, class_weight=class_weights)
model.reset_states()
return 0
a_train = np.random.normal(size=(50,24,5625))
y_train = a_train[:, -1, :]
a_train = a_train[:, :-1, :]
y_train[y_train > 0.] = 1.
y_train[y_train < 0.] = 0.
to_train(a_train, y_train)
我得到的错误是:
ValueError: `class_weight` must contain all classes in the data. The classes set([330]) exist in the data but not in `class_weight`.
'set([...])'中的值在每次运行时都会更改.但是正如我说的,数据中仅有的两个类是0和1.每个样本只有多个标签.因此,例如,一个响应(y_train)如下所示:
The value inside 'set([...])' changes at each run. But as I said, the only two classes in the data are 0 and 1; there are just multiple labels per sample. So for example, one response (y_train) looks like this:
print y_train[0,:]
#[ 0. 0. 1. ..., 0. 1. 0.]
在Keras中如何使用class_weights
解决多标签问题?
How can I use class_weights
for a multi-label problem in Keras?
推荐答案
是的.这是keras中的一个已知错误( issue#8011 ).基本上,在确定类的数量时,keras代码采用单热编码,而不是多标签序数编码.
Yep. It's a known bug in keras (issue #8011). Basically, keras code assumes one-hot encoding, when determines the number of classes, not multi-label ordinal encoding.
# if 2nd dimension is greater than 1, it must be one-hot encoded,
# so let's just get the max index...
if y.shape[1] > 1:
y_classes = y.argmax(axis=1)
除了设置y_true[:, 1] = 1
之外,我现在想不出更好的解决方法,即将y
中的1
位置保留"为始终为1.这将导致y_classes = 1
(这是二进制分类中正确的值).
I can't think of a better workaround right now, other than set y_true[:, 1] = 1
, i.e. "reserve" the 1
position in y
to be always one. This will cause y_classes = 1
(which is the right value in binary classification).
它为什么工作?当y_true[i]
获得带有一些前导零的[0, 0, ..., 0, 1, ...]
之类的值时,代码将失败. Keras的实现(错误地)通过max元素的索引估计类的数量,事实证明对于y[i][j] = 1
来说,它是某个j > 1
.这使Keras引擎认为存在超过2个类,因此提供的class_weights
是错误的.设置y_true[i][1] = 1
可以确保j <= 1
(因为np.argmax
选择最小的最大索引),这可以绕过keras防护.
Why does it work? The code fails when y_true[i]
gets the value like [0, 0, ..., 0, 1, ...]
with some number of leading zeros. Keras implementation (mistakenly) estimates the number of classes via the index of the max element, which turns out to be some j > 1
for which y[i][j] = 1
. This makes Keras engine think there are more than 2 classes, so the provided class_weights
are wrong. Setting y_true[i][1] = 1
makes sure that j <= 1
(because np.argmax
picks the smallest max index), which allows to bypass keras guards.
这篇关于多标签二进制分类中的Keras class_weight的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!