在 colab 上 - class_weight 导致 ValueError:具有多个元素的数组的真值不明确.使用 a.any() 或 a.all() [英] on colab - class_weight is causing a ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

查看:62
本文介绍了在 colab 上 - class_weight 导致 ValueError:具有多个元素的数组的真值不明确.使用 a.any() 或 a.all()的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在 google colab 上运行带有 keras 序列的 CNN.

i'm running a CNN with keras sequential on google colab.

我收到以下错误:ValueError:包含多个元素的数组的真值不明确.使用 a.any() 或 a.all()

i'm getting the following error: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

当我从 model.fit 函数中删除 class_weight 参数时,错误消失并且网络成功训练.但是,我真的想考虑不平衡的数据

when i remove the class_weight argument from the model.fit function, the error is gone and the network is trained succesfully. however, i really want to account for unbalanced data

我检查了我的 class_weights 向量的形状,它很好(和 nd.array,就像你从 sklearn 计算类权重函数生成 class_Weights 时得到的一样)

i checked the shape of my class_weights vector and it's good (and nd.array, just like you would get when generating class_Weights from sklearn compute class weights function )

不确定哪些细节是相关的,但我很乐意提供有关版本和所有这些混乱的更多细节.

not sure what details are relevant but i wil gladly provide more details regarding version and all that mess.

ps

一个可能很重要的事实——我的数据是 FER2013 数据,我使用的是 FERplus 标签.这意味着,我的样本与一个独特的类别无关,而是每个样本都有自己的每个类别的概率分布.最重要的是,我的标签是大小为 class_names 的向量,所有元素加起来为 1.

a fact that might be important - my data is the FER2013 data and i'm using FERplus labels. meaning, my samples are not associated with one unique class, rather each sample has it's own probability distribution for each class. bottom line, my labels are vectors of size class_names with all elements adding up to one.

为了超级清楚,举个例子:图像1标签 = [0,0,0,0,0.2,0,0.3,0,0,0.5]

just to be super clear, an example: img1 label = [0,0,0,0,0.2,0,0.3,0,0,0.5]

无论如何,我将 class_weights 计算为大小为 10 的 nd.array,元素范围在 0 和 1 之间,应该平衡更多代表的类.

anyhow, i computed class_weights as an nd.array of size 10 with elements ranging between 0 and 1, supposed to balance down the more represented classes.

我不确定这是否与错误有关,但我提出来以防万一.

i was not sure if that is relevant to the error, but i'm bringing it up just in case.

我的代码:

def create_model_plus():
  return tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(filters=32,kernel_size=5,strides=1,input_shape=(48, 48, 1),padding='same',use_bias=True,kernel_initializer='normal',bias_initializer=tf.keras.initializers.Constant(0.1),activation=tf.nn.relu),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), strides=2),
    tf.keras.layers.Conv2D(filters=64,kernel_size=5,strides=1,padding='same',use_bias=True,kernel_initializer='normal',bias_initializer=tf.keras.initializers.Constant(0.1),activation=tf.nn.relu),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), strides=1),
    tf.keras.layers.Conv2D(filters=128,kernel_size=5,strides=1,padding='same',use_bias=True,kernel_initializer='normal',bias_initializer=tf.keras.initializers.Constant(0.1),activation=tf.nn.relu),
    tf.keras.layers.BatchNormalization(),    
    tf.keras.layers.MaxPooling2D((2, 2), strides=1),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1008, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
  ])


history_df=[]
history_object=tf.keras.callbacks.History()
#save_best_object=tf.keras.callbacks.ModelCheckpoint('/Users/nimrodros', monitor='val_loss', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', period=1)

early_stop_object=tf.keras.callbacks.EarlyStopping(monitor='val_loss',min_delta=0.001, patience=4)
gony_adam=tf.keras.optimizers.Adam(
    lr=0.001
)
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.3,patience=3, min_lr=0.0001, verbose=1)

#log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
#tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

datagen = tf.keras.preprocessing.image.ImageDataGenerator(rotation_range=8, width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True,
    validation_split=0.3
    )
datagen.fit(images.reshape(28709,48,48,1))
model = create_model_plus()
model.compile(optimizer=gony_adam,
        loss='categorical_crossentropy',
        metrics=['accuracy'])
history = model.fit(x=datagen.flow(images.reshape(28709,48,48,1), FER_train_labels, batch_size=32,subset='training'),validation_data=datagen.flow(images.reshape(28709,48,48,1), FER_train_labels, batch_size=32,subset='validation'),steps_per_epoch=600,validation_steps=250,epochs=60,callbacks=[history_object,early_stop_object,reduce_lr],class_weight=cl_weigh)
history_df=pd.DataFrame(history.history)

希望有人知道该怎么做!谢谢!!!

hope someone knows what to do! thanks!!!

推荐答案

问题是 sklearn API 返回一个 numpy 数组,但 keras 需要字典作为 class_weight 的输入(请参阅 此处).您可以使用以下方法解决错误:

The problem is that the sklearn API returns a numpy array but the keras requires a dictionary as an input for class_weight (see here). You can resolve the error using below method:

from sklearn.utils import class_weight
weight = class_weight.compute_class_weight('balanced', np.unique(y_train), y_train)
weight = {i : weight[i] for i in range(5)}

这篇关于在 colab 上 - class_weight 导致 ValueError:具有多个元素的数组的真值不明确.使用 a.any() 或 a.all()的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
相关文章
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆