在 keras 中创建混合数据生成器(图像,csv) [英] Create a mixed data generator (images,csv) in keras
本文介绍了在 keras 中创建混合数据生成器(图像,csv)的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在构建一个具有多个输入的模型,如 pyimagesearch,但是我无法将所有图像加载到 RAM 中,我正在尝试创建一个使用 flow_from_directory
的生成器,并从 CSV 文件中获取每个图像的所有额外属性正在处理的图像.
I am building a model with multiple inputs as shown in pyimagesearch, however I can't load all images into RAM and I am trying to create a generator that uses flow_from_directory
and get from a CSV file all the extra attributes for each image being processed.
问题:如何从 CSV 中获取与图像生成器中每批图像对应的属性?
Question: How do I get the attributes from the CSV to correspond with the images in each batch from the image generator?
def get_combined_generator(images_dir, csv_dir, split, *args):
"""
Creates train/val generators on images and csv data.
Arguments:
images_dir : string
Path to a directory with subdirectories for each class.
csv_dir : string
Path to a directory containing train/val csv files with extra attributes.
split : string
Current split being used (train, val or test)
"""
img_width, img_height, batch_size = args
datagen = ImageDataGenerator(
rescale=1. / 255)
generator = datagen.flow_from_directory(
f'{images_dir}/{split}',
target_size=(img_width, img_height),
batch_size=batch_size,
shuffle=True,
class_mode='categorical')
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col='image')
def my_generator(image_gen, data):
while True:
i = image_gen.batch_index
batch = image_gen.batch_size
row = data[i * batch:(i + 1) * batch]
images, labels = image_gen.next()
yield [images, row], labels
csv_generator = my_generator(generator, df)
return csv_generator
推荐答案
我使用自定义生成器根据 Luke 的回答找到了一个解决方案
I found a solution based on Luke's answer using a custom generator
import random
import pandas as pd
import numpy as np
from glob import glob
from keras.preprocessing import image as krs_image
# Create the arguments for image preprocessing
data_gen_args = dict(
horizontal_flip=True,
brightness_range=[0.5, 1.5],
shear_range=10,
channel_shift_range=50,
rescale=1. / 255,
)
# Create an empty data generator
datagen = ImageDataGenerator()
# Read the image list and csv
image_file_list = glob(f'{images_dir}/{split}/**/*.JPG', recursive=True)
df = pd.read_csv(f'{csv_dir}/{split}.csv', index_col=csv_data[0])
random.shuffle(image_file_list)
def custom_generator(images_list, dataframe, batch_size):
i = 0
while True:
batch = {'images': [], 'csv': [], 'labels': []}
for b in range(batch_size):
if i == len(images_list):
i = 0
random.shuffle(images_list)
# Read image from list and convert to array
image_path = images_list[i]
image_name = os.path.basename(image_path).replace('.JPG', '')
image = krs_image.load_img(image_path, target_size=(img_height, img_width))
image = datagen.apply_transform(image, data_gen_args)
image = krs_image.img_to_array(image)
# Read data from csv using the name of current image
csv_row = dataframe.loc[image_name, :]
label = csv_row['class']
csv_features = csv_row.drop(labels='class')
batch['images'].append(image)
batch['csv'].append(csv_features)
batch['labels'].append(label)
i += 1
batch['images'] = np.array(batch['images'])
batch['csv'] = np.array(batch['csv'])
# Convert labels to categorical values
batch['labels'] = np.eye(num_classes)[batch['labels']]
yield [batch['images'], batch['csv']], batch['labels']
这篇关于在 keras 中创建混合数据生成器(图像,csv)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文