在 Keras 中构建数据时如何使用 repeat() 函数? [英] How to use repeat() function when building data in Keras?

问题描述

我正在用猫和狗的数据集训练一个二元分类器:
总数据集:10000 张图像
训练数据集:8000张图片
验证/测试数据集:2000 张图像

I am training a binary classifier on a dataset of cats and dogs:
Total Dataset: 10000 images
Training Dataset: 8000 images
Validation/Test Dataset: 2000 images

Jupyter 笔记本代码:

The Jupyter notebook code:

# Part 2 - Fitting the CNN to the images
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('dataset/training_set',
                                                 target_size = (64, 64),
                                                 batch_size = 32,
                                                 class_mode = 'binary')

test_set = test_datagen.flow_from_directory('dataset/test_set',
                                            target_size = (64, 64),
                                            batch_size = 32,
                                            class_mode = 'binary')

history = model.fit_generator(training_set,
                              steps_per_epoch=8000,
                              epochs=25,
                              validation_data=test_set,
                              validation_steps=2000)

我在 CPU 上训练它没有问题，但是当我在 GPU 上运行时，它抛出了这个错误:

I trained it on a CPU without a problem but when I run on GPU it throws me this error:

Found 8000 images belonging to 2 classes.
Found 2000 images belonging to 2 classes.
WARNING:tensorflow:From <ipython-input-8-140743827a71>:23: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 8000 steps, validate for 2000 steps
Epoch 1/25
 250/8000 [..............................] - ETA: 21:50 - loss: 7.6246 - accuracy: 0.5000
WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 200000 batches). You may need to use the repeat() function when building your dataset.
 250/8000 [..............................] - ETA: 21:52 - loss: 7.6246 - accuracy: 0.5000

我想知道如何在使用 Tensorflow 2.0 的 keras 中使用 repeat() 函数?

I would like to know how to use the repeat() function in keras using Tensorflow 2.0?

推荐答案

你的问题源于参数steps_per_epoch 和 validation_steps 需要等于总和划分为 batch_size 的数据点数.

Your problem stems from the fact that the parameters steps_per_epoch and validation_steps need to be equal to the total number of data points divided to the batch_size.

您的代码可以在 2017 年 8 月之前在 Keras 1.X 中运行.

Your code would work in Keras 1.X, prior to August 2017.

将您的 model.fit() 函数更改为:

Change your model.fit() function to:

history = model.fit_generator(training_set,
                              steps_per_epoch=int(8000/batch_size),
                              epochs=25,
                              validation_data=test_set,
                              validation_steps=int(2000/batch_size))

从 TensorFlow 2.1 开始，fit_generator() 已被弃用.您也可以在生成器上使用 .fit() 方法.

As of TensorFlow 2.1, fit_generator() is being deprecated. You can use .fit() method also on generators.

TensorFlow >= 2.1 代码:

history = model.fit(training_set.repeat(),
                    steps_per_epoch=int(8000/batch_size),
                    epochs=25,
                    validation_data=test_set.repeat(),
                    validation_steps=int(2000/batch_size))

注意int(8000/batch_size)等价于8000//batch_size(整数除法)

Notice that int(8000/batch_size) is equivalent to 8000 // batch_size (integer division)

这篇关于在 Keras 中构建数据时如何使用 repeat() 函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！