为 Keras model.fit_generator 使用生成器 [英] Use a generator for Keras model.fit_generator

问题描述

在编写自定义生成器来训练 Keras 模型时，我最初尝试使用 generator 语法.所以我从 __next__ 中yield.但是，当我尝试使用 model.fit_generator 训练我的模式时，我会收到一个错误，即我的生成器不是迭代器.解决方法是将 yield 更改为 return，这也需要重新调整 __next__ 的逻辑以跟踪状态.与让 yield 为我完成工作相比，这相当麻烦.

I originally tried to use generator syntax when writing a custom generator for training a Keras model. So I yielded from __next__. However, when I would try to train my mode with model.fit_generator I would get an error that my generator was not an iterator. The fix was to change yield to return which also necessitated rejiggering the logic of __next__ to track state. It's quite cumbersome compared to letting yield do the work for me.

有没有一种方法可以使用 yield 来完成这项工作?如果我必须使用 return 语句，我将需要编写更多的迭代器，这些迭代器必须具有非常笨拙的逻辑.

Is there a way I can make this work with yield? I will need to write several more iterators that will have to have very clunky logic if I have to use a return statement.

推荐答案

由于你没有发布它，我无法帮助你调试你的代码，但是我缩写了一个我为语义分割项目编写的自定义数据生成器，供你使用用作模板:

I can't help debug your code since you didn't post it, but I abbreviated a custom data generator I wrote for a semantic segmentation project for you to use as a template:

def generate_data(directory, batch_size):
    """Replaces Keras' native ImageDataGenerator."""
    i = 0
    file_list = os.listdir(directory)
    while True:
        image_batch = []
        for b in range(batch_size):
            if i == len(file_list):
                i = 0
                random.shuffle(file_list)
            sample = file_list[i]
            i += 1
            image = cv2.resize(cv2.imread(sample[0]), INPUT_SHAPE)
            image_batch.append((image.astype(float) - 128) / 128)

        yield np.array(image_batch)

用法:

model.fit_generator(
    generate_data('~/my_data', batch_size),
    steps_per_epoch=len(os.listdir('~/my_data')) // batch_size)

这篇关于为 Keras model.fit_generator 使用生成器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！