如何创建模块列表列表 [英] How to create a list of modulelists

问题描述

可以创建 PyTorch 模块列表的 python 列表吗?例如，如果我想在一个图层中有几个 Conv1d，然后在另一个具有不同 Conv1d 的图层.在每一层中，我需要根据层数对输出进行不同的操作.构建这个模块列表的python 列表"的正确方法是什么?

Is it ok to create a python-list of PyTorch modulelists? If for example, I want to have a few Conv1d in a layer and then another layer with different Conv1d. In each layer I need to do a different manipulation on the output depending on the layer number. What is the correct way to build this "python-list" of modulelists?

这样:

    class test(nn.Module):
        def __init__(...):
            self.modulelists = []
            for i in range(4):
                self.modulelists.append(nn.ModuleList([nn.Conv1d(10, 10, kernel_size=5) for _ in range(5)]))

或者这样:

    class test(nn.Module):
        def __init__(...):
            self.modulelists = nn.ModuleList()
            for i in range(4):
                self.modulelists.append(nn.ModuleList([nn.Conv1d(10, 10, kernel_size=5) for _ in range(5)]))

谢谢

推荐答案

您需要正确注册网络的所有子模块，以便 pytorch 可以访问它们的参数、缓冲区等.
只有使用正确的容器，才能做到这一点.
如果你将子模块存储在一个简单的 pythonic 列表中，pytorch 将不知道那里有子模块，它们将被忽略.

You need to register all sub-modules of your net properly so that pytorch can have access to their parameters, buffers etc.
This can be done only if you use proper containers.
If you store sub-modules in a simple pythonic list pytorch will have no idea there are sub modules there and they will be ignored.

所以，如果你使用简单的pythonic列表来存储子模块，例如当你调用model.cuda()时，列表中子模块的参数将不转移到GPU，而是留在CPU上.如果您调用 model.parameters() 将所有可训练参数传递给优化器，pytorch 将不会检测到所有子模块参数，因此优化器将不会看到"他们.

So, if you use simple pythonic list to store the sub-modules, when you call, for instance, model.cuda() the parameters of the sub-modules in the list will not be transferred to GPU, but rather remain on CPU. If you call model.parameters() to pass all trainable parameters to an optimizer, all the sub-modules parameters will not be detected by pytorch and thus the optimizer will not "see" them.

这篇关于如何创建模块列表列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！