ImageDataGenerator flow_from_dataframe multi_output回归和分类语法问题 [英] ImageDataGenerator flow_from_dataframe multi_output regression and classification syntax problem

问题描述

上下文

我正在使用tensorflow.keras.preprocessing.image ImageDataGenerator

来自tensorflow.keras.preprocessing.image的

 导入ImageDataGenerator 

我一直在努力寻找一个"multi_output"示例.定制生成器，将浮点向量(例如，代表边界框的4个向量)作为标签传递给2个网络头中的一个，并将一热编码的向量(例如3类)作为标签传递给另一头.>

第一个网络头将在4矢量边界框上进行回归

第一个头

然后，第二个网络头将对单热点"网络进行分类.3矢量.

第二个头

我的结构与

特别注意以下两行:

  y_col = [('sxu'，'syu'，'exu'，'eyu')，('cls_airplane'，'cls_face'，'cls_motorcycle')]，class_mode ="multi_output"， 

第一行应该指定一个包含2个标签元素的列表(我的模型中的2个头，4个向量的回归头和3个向量的分类头各一个)

下面显示了从CSV文件中加载到熊猫数据框中的示例

  id，文件，sx，sy，ex，ey，cls，sxu，syu，exu，eyu，w，h0，motorcycle.0001.jpg，31,19,233,141，摩托车，0.1183206106870229,0.11801242236024845,0.8893129770992366,0.8757763975155279,262,1611，motorcycle.0002.jpg，32,15,232,142，摩托车，0.12167300380228137,0.09259259259259259,0.8821292775665399,0.8765432098765432,263,162 

请注意，在上面的代码中，我添加了其他"one-hot"熊猫数据框的列(cls_motorcycle，cls_face，cls_airplane).这些是在我的"y_col"目录中的第二元组中引用的列.数组.

包含我的热门专栏的增强型熊猫数据框

错误

我正在使用tensorflow的keras实现.我收到的错误是密钥错误".进入熊猫数据框.

完整堆栈跟踪

显然，tensorflow keras不喜欢我为"multi_output"传递的元组列表.标签.它认为列表中的第一个元组实际上是一个键进入一列，而不是4个键进入4列.

问题

如何配置 flow_from_dataframe 带有"multi_output"一个头的回归标签，而第二个头的分类标签?

其他详细信息

我现在尝试执行以下操作(注意，我现在为标签生成一个2元组-这是我努力的最新状态)，我想我已经走了.(堆栈跟踪在代码段下方):

  def generate_image_generator(生成器，data_directory，data_items，target_size，类，batch_size，随机播放，class_mode):框架= []对于data_items中的di:df = pd.read_csv(data_directory + di [文件"])frame.append(df)df = pd.concat(帧)标签= ['sxu'，'syu'，'exu'，'eyu'，'cls_onehot']df ['cls_onehot'] = df ['cls'].str.get_dummies().values.tolist()genImages = generator.flow_from_dataframe(dataframe = df，directory = data_directory，target_size = target_size，x_col =文件"，y_col =标签，class_mode ="multi_output"，classes = classes，batch_size = batch_size，shuffle = shuffle，seed = 2)而True:图片，标签= genImages.next()产生图像[0]，([标签[0]，标签[1]，标签[2]，标签[3]]，标签[4]) 

我走得更远，所以也许我的生成器现在可以了，但是现在(当我开始训练模型时)我确实看到了以下踪迹:

第一次训练期间的新堆栈跟踪

在下面的pycharm调试视图中，您可以看到在生成器屈服"时存在的图像和标签.请注意，我的批次大小为5.

生成器生成的图像和标签批次

解决方案

这似乎是正确的答案:

  def generate_image_generator(生成器，data_directory，data_items，target_size，类，batch_size，随机播放，class_mode):框架= []对于data_items中的di:df = pd.read_csv(data_directory + di [文件"])frame.append(df)df = pd.concat(帧)df ['cls_onehot'] = df ['cls'].str.get_dummies().values.tolist()df ['bbox'] = df [['sxu'，'syu'，'exu'，'eyu']].values.tolist()genImages = generator.flow_from_dataframe(dataframe = df，directory = data_directory，target_size = target_size，x_col =文件"，y_col = ['bbox'，'cls_onehot']，class_mode ="multi_output"，classes = classes，batch_size = batch_size，shuffle = shuffle，seed = 2)而True:图片，标签= genImages.next()目标= {'class_label':标签[1]，'bounding_box':标签[0]}产生图像，目标 

The Context

I am using the tensorflow.keras.preprocessing.image ImageDataGenerator

from tensorflow.keras.preprocessing.image import ImageDataGenerator

I've struggled to find an example of a "multi_output" custom generator that passes a vector of floats (e.g. 4 vector representing a bounding box) as the label to one of the 2 network heads, and a one-hot encoded vector (e.g. 3 classes) as the label to the other head.

The first network head will perform regression on the 4-vector bounding box

First Head

and the second network head will perform classification on the "one-hot" 3-vector.

Second Head

I have a structure that is very similar to the one found here.

The only difference is that I dont want to load all the images into memory at once, hence my desire to use a generator.

I think my code is close, but the variety of examples I've found are not quite what I need.

This is what I first had (see Additional Details below for what i currently have):

def generate_image_generator(generator, data_directory, data_items, target_size, classes, batch_size, shuffle, class_mode):
frames=[]
for di in data_items:
    df = pd.read_csv(data_directory+di["file"])
    #df["cls"] = df["cls"].apply(lambda x: x.split(","))
    frames.append(df)
df = pd.concat(frames)
a = pd.get_dummies(df['cls'], prefix='cls')
df = pd.concat([df, pd.get_dummies(df['cls'], prefix='cls')], axis=1)
df.head()
#                                              y_col=(['sxu', 'syu', 'exu', 'eyu'], 'cls'),
genImages = generator.flow_from_dataframe(dataframe=df, directory=data_directory, target_size=target_size,
                                          x_col="file",
                                          y_col=[('sxu', 'syu', 'exu', 'eyu'), ('cls_airplane', 'cls_face', 'cls_motorcycle')],
                                          class_mode="multi_output",
                                          classes=classes, batch_size=batch_size, shuffle=shuffle, seed=2)

Notice in particular the following two lines:

 y_col=[('sxu', 'syu', 'exu', 'eyu'), ('cls_airplane', 'cls_face', 'cls_motorcycle')],
 class_mode="multi_output",

The first line is supposed to specify a list with 2 label elements (one for each of the 2 heads in my model, the 4-vector regression head, and the 3 vector classification head)

A sample from the CSV file that is loaded into the pandas dataframe is seen below

    id,file,sx,sy,ex,ey,cls,sxu,syu,exu,eyu,w,h
0,motorcycle.0001.jpg,31,19,233,141,motorcycle,0.1183206106870229,0.11801242236024845,0.8893129770992366,0.8757763975155279,262,161
1,motorcycle.0002.jpg,32,15,232,142,motorcycle,0.12167300380228137,0.09259259259259259,0.8821292775665399,0.8765432098765432,263,162

Notice that in the code above, I add additional "one-hot" columns (cls_motorcycle, cls_face, cls_airplane) to the pandas dataframe. These are the columns referenced in the second tuple that is in my "y_col" array.

The augmented pandas dataframe that includes my one-hot columns

The Error

I am using tensorflow's keras implementation. The error I receive is a "key error" into the pandas dataframe.

Full Stack Trace

Clearly tensorflow keras doesnt like the list of tuples I pass for the "multi_output" labels. It thinks the first tuple in the list is actually a single key into one column rather than 4 keys into 4 columns.

The Question

How can I configure flow_from_dataframe with a "multi_output" regression label for one head, and a classification label for the second head?

Additional Details

I have now tried to do the following (notice I now yield a 2-tuple for the label - this is the latest state of my effort), and I think I got further.. However I dont think this is right yet (stack trace below code snippet):

def generate_image_generator(generator, data_directory, data_items, target_size, classes, batch_size, shuffle, class_mode):
frames=[]
for di in data_items:
    df = pd.read_csv(data_directory+di["file"])
    frames.append(df)
df = pd.concat(frames)

labels = ['sxu', 'syu', 'exu', 'eyu', 'cls_onehot']
df['cls_onehot'] = df['cls'].str.get_dummies().values.tolist()

genImages = generator.flow_from_dataframe(dataframe=df, directory=data_directory, target_size=target_size,
                                          x_col="file",
                                          y_col=labels,
                                          class_mode="multi_output",
                                          classes=classes, batch_size=batch_size, shuffle=shuffle, seed=2)

while True:
    images, labels = genImages.next()
    yield images[0], ([labels[0], labels[1], labels[2], labels[3]], labels[4])

I get further, so perhaps my generator is ok now, but I do see the following trace now (when I start to train the model):

New Stack Trace During First Training Epoch

In the following pycharm debug view you can see the images and labels as they exist at the point that the generator "yields". Note, my batch size is 5.

Batch of Images and Labels yielded by generator

解决方案

This seems to be the right answer:

def generate_image_generator(generator, data_directory, data_items, target_size, classes, batch_size, shuffle, class_mode):
frames=[]
for di in data_items:
    df = pd.read_csv(data_directory+di["file"])
    frames.append(df)
df = pd.concat(frames)

df['cls_onehot'] = df['cls'].str.get_dummies().values.tolist()
df['bbox'] = df[['sxu', 'syu', 'exu', 'eyu']].values.tolist()

genImages = generator.flow_from_dataframe(dataframe=df, directory=data_directory, target_size=target_size,
                                          x_col="file",
                                          y_col=['bbox', 'cls_onehot'],
                                          class_mode="multi_output",
                                          classes=classes, batch_size=batch_size, shuffle=shuffle, seed=2)

while True:
    images, labels = genImages.next()
    targets = {
        'class_label': labels[1],
        'bounding_box': labels[0]
    }
    yield images, targets

这篇关于ImageDataGenerator flow_from_dataframe multi_output回归和分类语法问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！