ValueError: Shapes (None, 3, 2) 和 (None, 2) 不兼容使用 tfrecord [英] ValueError: Shapes (None, 3, 2) and (None, 2) are incompatible using tfrecord
问题描述
在下面的代码中,我将标签保存到 tfrecord 并再次读取.(实际上,我将图像和标签都保存到了 tfrecord,这是一个用于说明目的的简单示例).
In the following code, I save the label to tfrecord and read it again. (In reality, I save both images and labels to tfrecord, here is a simple example for illustration purpose) .
我收到一个错误 ValueError: Shapes (None, 3, 2) and (None, 2) are incompatible
,我应该如何解决这个问题?我正在使用 Tensorflow 2.3.关键部分应该在parse_examples
的return语句中.
I got an error ValueError: Shapes (None, 3, 2) and (None, 2) are incompatible
, how should I fix this? I am using Tensorflow 2.3. The key part should be in the return statement of parse_examples
.
import contextlib2
import numpy as np
import tensorflow as tf
from tensorflow.keras import Model
from tensorflow.keras.layers import GlobalAveragePooling2D, Dense, Dropout
def process_image():
dic={
"image/label": tf.train.Feature(int64_list=tf.train.Int64List(value=[0,1]))
}
return tf.train.Example(features=tf.train.Features(feature=dic))
with contextlib2.ExitStack() as tf_record_close_stack:
output_tfrecords = [tf_record_close_stack.enter_context(tf.io.TFRecordWriter(file_name)) for file_name in
[f"data_train.tfrecord"]]
output_tfrecords[0].write(process_image().SerializeToString())
def parse_examples(examples):
parsed_examples = tf.io.parse_example(examples, features={
"image/label": tf.io.FixedLenFeature(shape=[2], dtype=tf.int64),
})
res = np.random.randint(2, size=3072).reshape(32, 32, 3)
return (res, [parsed_examples["image/label"],parsed_examples["image/label"],parsed_examples["image/label"]])
def process_dataset(dataset):
dataset = dataset.map(parse_examples, num_parallel_calls=tf.data.experimental.AUTOTUNE)
dataset = dataset.batch(1)
return dataset
train_data = tf.data.TFRecordDataset(filenames="data_train.tfrecord")
train_data = process_dataset(train_data)
base_model = tf.keras.applications.EfficientNetB7(input_shape=(32,32, 3), weights='imagenet',
include_top=False) # or weights='noisy-student'
for layer in base_model.layers[:]:
layer.trainable = False
x = GlobalAveragePooling2D()(base_model.output)
dropout_rate = 0.3
x = Dense(256, activation='relu')(x)
x = Dropout(dropout_rate)(x)
x = Dense(256, activation='relu')(x)
x = Dropout(dropout_rate)(x)
all_target = []
loss_list = []
test_metrics = {}
for name, node in [("task1", 2), ("task2", 2), ("task3", 2)]:
y1 = Dense(128, activation='relu')(x)
y1 = Dropout(dropout_rate)(y1)
y1 = Dense(64, activation='relu')(y1)
y1 = Dropout(dropout_rate)(y1)
y1 = Dense(node, activation='softmax', name=name)(y1)
all_target.append(y1)
loss_list.append('categorical_crossentropy')
test_metrics[name] = "accuracy"
# model = Model(inputs=model_input, outputs=[y1, y2, y3])
model = Model(inputs=base_model.input, outputs=all_target)
model.compile(loss=loss_list, optimizer='adam', metrics=test_metrics)
history = model.fit(train_data, epochs=1, verbose=1)
推荐答案
事实证明,只需更改 parse_examples
中的 return
语句即可:
It turns out that, just change the return
statement from parse_examples
works:
return (res, {"task1":parsed_examples["image/label"],"task2":parsed_examples["image/label"],"task3":parsed_examples["image/label"]})
其中task1
,task2
,task3
是我给的softmax层的名字.
Where task1
,task2
,task3
are the names of the softmax layers given by me.
这篇关于ValueError: Shapes (None, 3, 2) 和 (None, 2) 不兼容使用 tfrecord的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!