火炬损失函数尺寸不匹配 [英] Pytorch loss function dimensions do not match

问题描述

我正尝试使用批量培训来运行单词嵌入，如下所示.

I'm trying to run word embeddings using batch training, as shown below.

def forward(self, inputs):
    print(inputs.shape)
    embeds = self.embeddings(inputs)
    print(embeds.shape)
    out = self.linear1(embeds)
    print(out.shape)
    out = self.activation_function1(out)
    print(out.shape)
    out = self.linear2(out).cuda()
    print(out.shape)
    out = self.activation_function2(out)
    print(out.shape)
    return out.cuda()

在这里，我使用的是上下文大小4，批处理大小32，嵌入大小50，隐藏层大小64，vocab大小9927

Here, I'm using context size 4, batch size 32, embedding size 50, hidden layer size 64, vocab size 9927

形状"函数的输出为

print(inputs.shape)----> torch.Size([4，32])

print(inputs.shape) ----> torch.Size([4, 32])

print(embeds.shape)----> torch.Size([4，32，50])

print(embeds.shape) ----> torch.Size([4, 32, 50])

print(out.shape)----> torch.Size([4，32，64])

print(out.shape) ----> torch.Size([4, 32, 64])

print(out.shape)----> torch.Size([4，32，64])

print(out.shape) ----> torch.Size([4, 32, 64])

print(out.shape)----> torch.Size([4，32，9927])

print(out.shape) ----> torch.Size([4, 32, 9927])

print(out.shape)----> torch.Size([4，32，9927])

print(out.shape) ----> torch.Size([4, 32, 9927])

这些形状正确吗?我很困惑.

Are the shapes of these correct? I'm quite confused.

此外，当我训练时，它会返回错误:

Also, when I train, it returns an error:

def train(epoch):
  model.train()
  for batch_idx, (data, target) in enumerate(train_loader, 0):
    optimizer.zero_grad()
    output = model(torch.stack(data))
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()

我在损失=标准(输出，目标)"行中出错.它说:预期输入batch_size(4)以匹配目标batch_size(32)."我的前进"功能的形状正确吗?我对批量培训不太熟悉.如何使尺寸匹配?

I'm getting an error in the line "loss = criterion(output, target)". It says "Expected input batch_size (4) to match target batch_size (32)." Are my shapes for the "forward" function correct? I'm not that familiar with batch training. How do I make the dimensions match?

---------在下面发布初始化代码-----

------- Posting init code below -----

  def __init__(self, vocab_size, embedding_dim):
    super(CBOW, self).__init__()
    self.embeddings = nn.Embedding(vocab_size, embedding_dim)
    self.linear1 = nn.Linear(embedding_dim, 64)
    self.activation_function1 = nn.ReLU()
    self.linear2 = nn.Linear(64, vocab_size)
    self.activation_function2 = nn.LogSoftmax(dim = -1)

推荐答案

torch.nn.Linear的forward方法需要将批量大小作为第一个参数.

torch.nn.Linear's forward method needs batch size as first argument.

您正在以第二个(首先是时间步长)的形式提供它，请使用permute(1, 0, 2)使其成为第一个.

You are supplying it as second (first being timesteps), use permute(1, 0, 2) to make them first.

此外，线性层通常采用2D输入，第一个是批处理，第二个是输入尺寸.您的语言是3d(因为我认为是单词)，也许您想使用递归神经网络(例如torch.nn.LSTM)?

Furthermore, linear layers usually take 2D input, with first being batch and second being dimension of input. Yours is 3d because of words (I assume), maybe you want to use recurrent neural networks (e.g. torch.nn.LSTM)?

这篇关于火炬损失函数尺寸不匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！