Keras ImageDataGenerator:如何在图像路径中使用数据增强 [英] Keras ImageDataGenerator : how to use data augmentation with images paths

问题描述

我正在研究CNN模型，我想使用一些数据扩充功能，但是会出现两个问题:

I am working on a CNN model and I would like to use some data augmentation, but two problems arise :

1. 我的标签是图像(我的模型是某种自动编码器，但是预期的输出图像与输入的图像不同)，因此我不能使用诸如 ImageDataGenerator.flow_from_directory().我当时在想 ImageDataGenerator.flow(train_list，y = labels_list)，但是我遇到了第二个问题:
2. 我的输入和标签数据集都非常庞大，我更喜欢使用图像路径( flow()函数无法正确处理)，而不是将所有数据集加载到单个数组中，并使RAM爆炸.

1. My labels are images (my model is some kind of autoencoder, but the expected output images are different from my input images), thus I cannot use functions such as ImageDataGenerator.flow_from_directory(). I was thinking of ImageDataGenerator.flow(train_list, y = labels_list), but there comes my second issue :
2. Both my input and labels datasets being really huge, I'd prefer working with images paths (which are not handled correctly by the flow() function) rather than loading all my dataset in a single array and making my RAM explode.

如何正确处理这两个问题?对于我发现的情况，可能有两种解决方案:

How can I properly deal with these two issues? For what I've found, there might be two solutions :

1. 创建我自己的生成器:我听说过 Sequence 类中的Keras __ getitem __ 函数，但这会影响 ImageDataGenerator 类?
2. 使用TF DATA或TFRecords ，但是它们似乎很难使用，并且数据增强仍有待实现.

1. Create my own generator : I've heard of the Keras __getitem__ function in the Sequence class, but can it impact the ImageDataGenerator class?
2. Work with TF DATA or TFRecords, but they seem pretty difficult to use, and the data augmentation is still to be implemented.

有没有最简单的方法来克服这个简单的问题?一个简单的技巧就是强制 ImageDataGenerator.flow()使用nparray的图像路径而不是nparray的图像，但是我担心修改Keras/tensorflow文件将产生意想不到的结果(由于某些函数在其他类中被调用，因此局部更改可能很快导致我的所有笔记本库中发生全局更改).

Is there an easiest way to overcome this simple problem? A mere trick would be to force ImageDataGenerator.flow() to work with a nparray of images paths rather than a nparray of images, but I fear that modifying the Keras/tensorflow files will have unexpected consequences (as some functions are called in other classes, a local change can soon result in a global change in all of my notebook library).

推荐答案

好，所以我终于找到了如何解决这些问题的原因，这要归功于

Ok so I finally found out how to deal with these issues thanks to this article. My mistake was that I kept using ImageDataGenerator despite its lack of flexibility, the solution is thus simple : use another data augmentation tool.

我们可以按以下方式恢复作者的方法:

We can resume the author's method as following :

1. 首先，创建一个个性化的批处理生成器作为Keras Sequence 类的子类(这意味着实现一个用于加载图像的 __ getitem __ 函数根据它们各自的路径.
2. 使用数据扩充分配库.它的优点是可以提供更多的转换功能，例如 Imgaug 或 ImageDataGenerator ，同时速度更快.此外，此网站允许您测试其某些增强方法，甚至使用您自己的图像！请参阅此以获取详尽的列表.

1. First, create a personalized batch generator as a subclass of Keras Sequence class (which implies to implement a __getitem__ function that loads the images according to their respective paths).
2. Use the data augmentation albumentations library. It has the advantages of offering more transformation functions as Imgaug or ImageDataGenerator, while being faster. Moreover, this website allows you to test some of its augmentation methods, even with your own images ! See this one for the exhaustive list.

---

该库的缺点是，由于它相对较新，因此在网上找不到很少的文档，而且我花了数小时试图解决遇到的问题.

---

The drawback of this library is that, as it is relatively new, few documentation can be found online, and I've spent several hours trying to resolve an issue I encountered.

实际上，当我尝试可视化某些增强功能时，结果是全黑图像(奇怪的事实:只有当我使用 RandomGamma 或 RandomBrightnessContrast .使用诸如 Horizo​​ntalFlip 或 ShiftScaleRotate 之类的转换功能，它将正常工作.)

Indeed, when I tried to visualize some augmentation functions, the results were entirely black images (strange fact : these would happen only when I was modifying the intensity of the pixels, with methods like RandomGamma or RandomBrightnessContrast. With transformation functions such as HorizontalFlip or ShiftScaleRotate, it would work normally).

在一整天的尝试后，我终于找到了这个解决方案，如果您尝试使用该库，可能会为您提供帮助:必须完成图像的加载使用OpenCV (我正在使用 tf.keras.preprocessing.image 中的 load_img 和 img_to_array 函数进行加载和处理).如果有人能解释为什么它不起作用，我将很高兴听到它.

After an entire half day of trying-to-find-what's-wrong, I eventually came up with this solution, that might help you if you were to try this library : the loading of images has to be done with OpenCV (I was using load_img and img_to_array functions from tf.keras.preprocessing.image for the loading and processing). If anyone has an explanation of why this doesn't work, I'd be glad to hear it.

无论如何，这是我显示增强图像的最终代码:

Anyway, here is my final code to display an augmented image :

!pip install -U git+https://github.com/albu/albumentations > /dev/null && echo "All libraries are successfully installed!"
from albumentations import Compose, HorizontalFlip, RandomBrightnessContrast, ToFloat, RGBShift
import cv2
import matplotlib.pyplot as plt
import numpy as np
from google.colab.patches import cv2_imshow # I work on a Google Colab, thus I cannot use cv2.imshow()


augmentation = Compose([HorizontalFlip(p = 0.5),
                        RandomBrightnessContrast(p = 1),
                        ToFloat(max_value = 255) # Normalize the pixels values into the [0,1] interval
                        # Feel free to add more !
                        ])

img = cv2.imread('Your_path_here.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # cv2.imread() loads the images in BGR format, thus you have to convert it to RGB before applying any transformation function.
img = augmentation(image = img)['image'] # Apply the augmentation functions to the image.
plt.figure(figsize=(7, 7))
plt.imshow((img*255).astype(np.uint8)) # Put the pixels values back to [0,255]. Replace by plt.imshow(img) if the ToFloat function is not used.
plt.show()


'''
If you want to display using cv2_imshow(), simply replace the last three lines by :

img = cv2.normalize(img, None, 255,0, cv2.NORM_MINMAX, cv2.CV_8UC1) # if the ToFloat argument is set up inside Compose(), you have to put the pixels values back to [0,255] before plotting them with cv2_imshow(). I couldn't try with cv2.imshow(), but according to the documentation it seems this line would be useless with this displaying function.
cv2_imshow(img)

I don't recommend it though, because cv2_imshow() plot the images in BGR format, thus some augmentation methods such as RGBShift will not work properly.
'''

---

我在 albumentations 库中遇到了一些问题(我在这个问题，但目前我仍然没有答案)，因此我建议使用 Imgaug 进行数据增强:它工作正常，几乎与 albumentations 一样容易使用，尽管可用的转换函数要少得多.

---

EDIT :

I've encountered several issues with the albumentations library (that I described in this question on Github, but for now I still have had no answers) thus I'd better recommend using Imgaug for your data augmentation : it works just fine and is almost as easy to use as albumentations, even though there is a little bit less available transformation functions.

这篇关于Keras ImageDataGenerator:如何在图像路径中使用数据增强的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！