PyTorch-contiguous() [英] PyTorch - contiguous()

问题描述

我正在github上查看LSTM语言模型的示例

I was going through this example of a LSTM language model on github (link). What it does in general is pretty clear to me. But I'm still struggling to understand what calling contiguous() does, which occurs several times in the code.

例如，在代码输入的第74/75行中，创建了LSTM的目标序列. 数据(存储在ids中)为二维，其中第一维为批处理大小.

For example in line 74/75 of the code input and target sequences of the LSTM are created. Data (stored in ids) is 2-dimensional where first dimension is the batch size.

for i in range(0, ids.size(1) - seq_length, seq_length):
    # Get batch inputs and targets
    inputs = Variable(ids[:, i:i+seq_length])
    targets = Variable(ids[:, (i+1):(i+1)+seq_length].contiguous())

举一个简单的例子，当使用批处理大小1和seq_length 10 inputs和targets时，如下所示:

So as a simple example, when using batch size 1 and seq_length 10 inputs and targets looks like this:

inputs Variable containing:
0     1     2     3     4     5     6     7     8     9
[torch.LongTensor of size 1x10]

targets Variable containing:
1     2     3     4     5     6     7     8     9    10
[torch.LongTensor of size 1x10]

所以总的来说，我的问题是，contiguous()是什么，为什么我需要它?

So in general my question is, what does contiguous() and why do I need it?

此外，我不明白为什么要为目标序列而不是输入序列调用该方法，因为两个变量都包含相同的数据.

Further I don't understand why the method is called for the target sequence and but not the input sequence as both variables are comprised of the same data.

targets如何不连续而inputs仍然连续?

How could targets be uncontiguous and inputs still be contiguous?

我试图省略调用contiguous()的操作，但这在计算损失时会导致出现错误消息.

I tried to leave out calling contiguous(), but this leads to an error message when computing the loss.

RuntimeError: invalid argument 1: input is not contiguous at .../src/torch/lib/TH/generic/THTensor.c:231

因此显然在此示例中调用contiguous()是必需的.

So obviously calling contiguous() in this example is necessary.

(为了保持可读性，我避免在此处发布完整的代码，可以使用上面的GitHub链接找到它.)

(For keeping this readable I avoided posting the full code here, it can be found by using the GitHub link above.)

提前谢谢！

推荐答案

在PyTorch中对Tensor进行的操作很少真正改变张量的内容，而只是如何将索引转换为张量到字节位置.这些操作包括:

There are few operations on Tensor in PyTorch that do not really change the content of the tensor, but only how to convert indices in to tensor to byte location. These operations include:

narrow()，view()，expand()和transpose()

例如:当您调用transpose()时，PyTorch不会生成具有新布局的新张量，它只是修改Tensor对象中的元信息，因此偏移和步幅是用于新形状的.转置张量和原始张量确实共享内存！

For example: when you call transpose(), PyTorch doesn't generate new tensor with new layout, it just modifies meta information in Tensor object so offset and stride are for new shape. The transposed tensor and original tensor are indeed sharing the memory!

x = torch.randn(3,2)
y = torch.transpose(x, 0, 1)
x[0, 0] = 42
print(y[0,0])
# prints 42

这是连续概念的来源.x上方是连续的，但y并不是因为其内存布局不同于从头制作的相同形状的张量.请注意，单词连续" 有点误导，因为它不是张量的内容散布在未连接的内存块周围.这里字节仍然分配在一个内存块中，但是元素的顺序不同！

This is where the concept of contiguous comes in. Above x is contiguous but y is not because its memory layout is different than a tensor of same shape made from scratch. Note that the word "contiguous" is bit misleading because its not that the content of tensor is spread out around disconnected blocks of memory. Here bytes are still allocated in one block of memory but the order of the elements is different!

调用contiguous()时，它实际上会复制张量，因此元素的顺序将与从头开始创建相同形状的张量相同.

When you call contiguous(), it actually makes a copy of tensor so the order of elements would be same as if tensor of same shape created from scratch.

通常，您不必为此担心.如果PyTorch期望连续张量，但如果不是，那么您将得到RuntimeError: input is not contiguous，然后只需将调用添加到contiguous().

Normally you don't need to worry about this. If PyTorch expects contiguous tensor but if its not then you will get RuntimeError: input is not contiguous and then you just add a call to contiguous().

这篇关于PyTorch-contiguous()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！