Keras/Tensorflow中的类生成器(继承序列)线程安全吗? [英] Is the class generator (inheriting Sequence) thread safe in Keras/Tensorflow?

问题描述

为使模型的训练更快，在CPU上填充/生成批次并在GPU上并行运行模型的训练似乎是一个好习惯.为此，可以使用Python编写一个生成器类，该类继承Sequence类.

For making the training of a model faster, it seems to be a good practice to populate/generate batches on CPU and run the training of the model on GPU in parallel. For this purpose, a generator class can be written in Python that inherits the Sequence class.

以下是文档的链接: https://www.tensorflow.org/api_docs/python/tf/keras /utils/Sequence

Here is the link to the documentation: https://www.tensorflow.org/api_docs/python/tf/keras/utils/Sequence

文档指出的重要内容是:

The important thing that the document states is:

Sequence是执行多处理的更安全方法.这种结构 保证网络仅对每个样本进行一次训练 生成器不是这样的时代.

Sequence are a safer way to do multiprocessing. This structure guarantees that the network will only train once on each sample per epoch which is not the case with generators.

它给出了一个简单的代码示例，如下所示:

And it gives a simple code example as following:

from skimage.io import imread
from skimage.transform import resize
import numpy as np
import math

# Here, `x_set` is list of path to the images
# and `y_set` are the associated classes.

class CIFAR10Sequence(Sequence):

    def __init__(self, x_set, y_set, batch_size):
        self.x, self.y = x_set, y_set
        self.batch_size = batch_size

    def __len__(self):
        return math.ceil(len(self.x) / self.batch_size)

    def __getitem__(self, idx):
        batch_x = self.x[idx * self.batch_size:(idx + 1) *
        self.batch_size]
        batch_y = self.y[idx * self.batch_size:(idx + 1) *
        self.batch_size]

        return np.array([
            resize(imread(file_name), (200, 200))
               for file_name in batch_x]), np.array(batch_y)

在我看来，在模型中理想地要做的是创建此生成器类的实例并将其提供给fit_generator(...)函数.

What - to my understanding - ideally needs to be done in the model is to create an instance of this generator class and give it to the fit_generator(...) function.

gen = CIFAR10Sequence(x_set, y_set, batch_size)
# Train the model
model.fit_generator(generator=gen,
                    use_multiprocessing=True,
                    workers=6)

以下是Keras文档的引文:

Here is a quote from Keras documentation:

使用keras.utils.Sequence保证订购并保证 使用时每个历元的每次输入的单次使用 use_multiprocessing=True .

The use of keras.utils.Sequence guarantees the ordering and guarantees the single use of every input per epoch when using use_multiprocessing=True.

在这种形状下，我认为此设置是线程安全的. 问题1):我的假设正确吗?

In this shape, I assume that this setup is thread safe. Question 1) Is my assumption correct?

一个令人困惑的事情是，在Windows 10上可能未将参数use_multiprocessing设置为True.似乎只能在Linux上将其设置为True. (我不知道在其他平台上情况如何.)但是workers参数仍然可以设置为大于0的值.

One confusing thing though is that the parameter use_multiprocessing may not be set to True on Windows 10. Keras does not allow it; seemingly it only can be set to True on Linux. (I don't know how it is in other platforms.) But the workers parameter can still be set to a value that is greater than 0.

让我们看一下这两个参数的定义:

Let's have a look at the definition of these 2 parameters:

workers: 整数.使用时最多可启动的进程数 基于进程的线程.如果未指定，则worker将默认为1. 0，将在主线程上执行生成器.

workers: Integer. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.

use_multiprocessing: 布尔值.如果为True，则使用基于进程的线程.如果 未指定，use_multiprocessing将默认为False.注意 因为此实现依赖于多处理，所以您不应 将不可拾取的参数传递给生成器，因为它们无法传递 轻松进行子进程.

use_multiprocessing: Boolean. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can't be passed easily to children processes.

因此，通过使用workers参数，似乎有可能创建多个进程来加快训练速度，而与use_multiprocessing是否为True无关.

So, by using the workers parameter, it seems to be possible to create multiple processes to speed up the training independent from whether use_multiprocessing is True or not.

如果要使用继承Sequence的生成器类(在Windows 10上为 )，则必须将use_multiprocessing设置为False，如下所示:

If one wants to use the generator class inheriting Sequence (on Windows 10), s/he has to set the use_multiprocessing to False as following:

gen = CIFAR10Sequence(x_set, y_set, batch_size)
# Train the model
model.fit_generator(generator=gen,
                    use_multiprocessing=False,  # CHANGED
                    workers=6)

由于worker = 6，这里仍然有多个进程在运行.

And there are still multiple processes running here because workers = 6.

问题2):此设置是否仍然是线程安全的?或者在将use_multiprocessing参数设置为False之后，线程安全特性现在丢失了吗?根据文档，我无法弄清楚.

Question 2) Is this setup still thread safe or is the thread safe characteristic lost now after setting the use_multiprocessing parameter to False? I cannot make it clear based on documentation.

问题3)仍然与该主题相关...当以这种方式进行训练时，CPU生成数据并在GPU上进行训练，如果要训练的模型很浅，由于GPU一直在等待来自CPU的数据，因此GPU的使用率最终非常低，CPU的使用率却显着提高.在这种情况下，是否有办法利用一些GPU资源进行数据生成?

Question 3) Still related to this topic... When training is done in this way where data is generated by the CPU and training on GPU, if the model that is being trained is shallow, the GPU utilization ends up being very low and CPU utilization becomes significantly higher because the GPU keeps waiting for data that will come from CPU. In such cases, is there a way to utilize some GPU resources as well for data generation?

推荐答案

在看过这篇文章的人中，似乎没有人给出最终答案，因此我想给出对我有用的答案.由于该领域缺乏文档，我的答案可能缺少一些相关细节.请随时添加我在这里没有提及的更多信息.

Among those who have seen this post, no one seems to have the ultimate answer so that I wanted to give my answer that worked out for me. Because of lack of documentation in the domain, my answer might be missing some relevant details. Please feel free to add more information that I do not mention down here.

貌似，在 Windows 中不支持使用Python编写继承Sequence类的生成器类. (您似乎可以使其在Linux上运行.)要使其运行，您需要设置参数use_multiprocessing=True(使用类方法).但是如上所述，它在Windows上不起作用，因此您必须将use_multiprocessing设置为False(在Windows上). 但是，这并不意味着多重处理在Windows上不起作用.即使设置了use_multiprocessing=False，使用以下设置运行代码时仍可以支持多处理，只需将workers参数设置为大于1的任何值即可.

Seemingly, writing a generator class in Python that inherits the Sequence class is just not supported in Windows. (You can seemingly make it work on Linux.) To be able to make it work, you need to set the parameter use_multiprocessing=True (with the class approach). But it is not working on Windows as mentioned so that you have to set use_multiprocessing to False (on Windows). Nevertheless, that does not mean that multiprocessing does not work on Windows. Even if you set use_multiprocessing=False, multiprocessing can still be supported when the code is run with the following setup where you just set the workers parameter to any value that is bigger than 1.

示例:

history = \
   merged_model.fit_generator(generator=train_generator,
                              steps_per_epoch=trainset_steps_per_epoch,
                              epochs=300,
                              verbose=1,
                              use_multiprocessing=False,
                              workers=3,
                              max_queue_size=4)

在这一点上，让我们再次记住Keras文档:

At this point, let's remember the Keras documentation again:

使用keras.utils.Sequence保证顺序并保证 使用时每个时期每个输入的单次使用 use_multiprocessing = True.

The use of keras.utils.Sequence guarantees the ordering and guarantees the single use of every input per epoch when using use_multiprocessing=True.

据我了解，如果为use_multiprocessing=False，则生成器不再是线程安全的，这使得编写继承Sequence的 generator类变得困难.

To my understanding, if use_multiprocessing=False, then the generator is not thread safe anymore, which makes it difficult to write a generator class that inherits Sequence.

要解决此问题，我自己编写了一个生成器，并手动将其设为线程安全.这是一个示例伪代码:

To come around this problem, I have written a generator myself which I have made thread safe manually. Here is an example pseudocode:

import tensorflow as tf
import threading

class threadsafe_iter:
    """Takes an iterator/generator and makes it thread-safe by
    serializing call to the `next` method of given iterator/generator.
    """
    def __init__(self, it):
        self.it = it
        self.lock = threading.Lock()

    def __iter__(self):
        return self

    def __next__(self): # Py3
        return next(self.it)

    #def next(self):     # Python2 only
    #    with self.lock:
    #        return self.it.next()

def threadsafe_generator(f):
    """A decorator that takes a generator function and makes it thread-safe.
    """
    def g(*a, **kw):
        return threadsafe_iter(f(*a, **kw))
    return g


@threadsafe_generator
def generate_data(tfrecord_file_path_list, ...):

    dataset = tf.data.TFRecordDataset(tfrecord_file_path_list)

    # example proto decode
    def _parse_function(example_proto):
      ...
      return batch_data

    # Parse the record into tensors.
    dataset = dataset.map(_parse_function)  

    dataset = dataset.shuffle(buffer_size=100000)

    # Repeat the input indefinitly
    dataset = dataset.repeat()  

    # Generate batches
    dataset = dataset.batch(batch_size)

    # Create an initializable iterator
    iterator = dataset.make_initializable_iterator()

    # Get batch data
    batch_data = iterator.get_next()

    iterator_init_op = iterator.make_initializer(dataset)

    with tf.Session() as sess:

        sess.run(iterator_init_op)

        while True:            
            try:
                batch_data = sess.run(batch_data)
            except tf.errors.OutOfRangeError:
                break
            yield batch_data

好吧，可以这样进行讨论是否真的很优雅，但似乎运行得很好.

Well, it can be discussed if it is really elegant to do it in this way but it seems to be working pretty well.

总结:

• 如果在Windows上编写程序，请将use_multiprocessing设置为False.
• (据我所知，到今天为止)不支持编写在Windows上编写代码时继承Sequence的生成器类. (我猜这是一个Tensorflow/Keras问题).
• 要解决此问题，请编写一个普通的生成器，使生成器线程安全，并将workers设置为大于1的数字.

• If writing your program on Windows, set use_multiprocessing to False.
• (As of today, to my knowledge) it is not supported to write a generator class that inherits Sequence when writing code on Windows. (It is a Tensorflow/Keras problem I guess).
• To come around the problem, write an ordinary generator, make your generator thread safe, and set workers to a number that is greater than 1.

重要说明::在此设置中，生成器在CPU上运行，而训练在GPU上进行.我可能会观察到的一个问题是，如果您正在训练的模型足够浅，那么GPU的利用率仍然很低，而CPU利用率却很高.如果模型较浅并且数据集足够小，那么将所有数据存储在内存中并在GPU上运行所有数据可能是一个不错的选择.它应该大大加快培训的速度.如果出于任何原因想要同时使用CPU和GPU，我的建议是尝试使用Tensorflow的tf.data API，该API可显着加快数据预处理和批处理的速度.如果生成器仅使用Python编写，则GPU会一直等待数据以继续进行训练.可以说有关Tensorflow/Keras文档的所有内容，但这确实是高效的代码！

Important note: In this setup, the generator is being run on CPU and the training is being done on GPU. One problem I could observe is that if the model you are training is shallow enough, the utilization of GPU remains very low while CPU utilization gets high. If the model is shallow and the dataset is small enough, it can be a good option to store all the data in the memory and run everything on GPU. It should speed up the training significantly. If, for any reason, you would like to use CPU and GPU simultaneously, my modest recommendation is to try to use Tensorflow's tf.data API which significantly speeds up the data preprocessing and batch preparation. If the generator is only written in Python, GPU keeps waiting for data to continue with the training. One can say everything about the Tensorflow/Keras documentation, but it is really efficient code!

任何对API有更全面的了解的人，请看这篇文章，如果我误解了任何内容，或者更新了API甚至在Windows上也解决了问题，请随时在这里纠正我.

Anyone having more complete knowledge on the API and seeing this post, please feel free to correct me here in case I misunerstand anything or the API is updated to solve the problems even on Windows.

这篇关于Keras/Tensorflow中的类生成器(继承序列)线程安全吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！