如何将 HuggingFace 的 Seq2seq 模型转换为 onnx 格式 [英] how to convert HuggingFace's Seq2seq models to onnx format

问题描述

我正在尝试将 HuggingFace 的转换器模型中的 Pegasus 新闻编辑室转换为 ONNX 格式.我跟着 这份指南由 Huggingface 出版.安装先决条件后，我运行了以下代码:

I am trying to convert the Pegasus newsroom in HuggingFace's transformers model to the ONNX format. I followed this guide published by Huggingface. After installing the prereqs, I ran this code:

!rm -rf onnx/
from pathlib import Path
from transformers.convert_graph_to_onnx import convert

convert(framework="pt", model="google/pegasus-newsroom", output=Path("onnx/google/pegasus-newsroom.onnx"), opset=11)

并得到这些错误:

ValueError                                Traceback (most recent call last)
<ipython-input-9-3b37ed1ceda5> in <module>()
      3 from transformers.convert_graph_to_onnx import convert
      4 
----> 5 convert(framework="pt", model="google/pegasus-newsroom", output=Path("onnx/google/pegasus-newsroom.onnx"), opset=11)
      6 
      7 

6 frames
/usr/local/lib/python3.6/dist-packages/transformers/models/pegasus/modeling_pegasus.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, head_mask, encoder_head_mask, past_key_values, inputs_embeds, use_cache, output_attentions, output_hidden_states, return_dict)
    938             input_shape = inputs_embeds.size()[:-1]
    939         else:
--> 940             raise ValueError("You have to specify either decoder_input_ids or decoder_inputs_embeds")
    941 
    942         # past_key_values_length

ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

我以前从未见过此错误.有什么想法吗?

I have never seen this error before. Any ideas?

推荐答案

Pegasus 是一个 seq2seq 模型，您不能使用此方法直接转换 seq2seq 模型(编码器-解码器模型).guide 用于 BERT，它是一种编码器模型.使用此方法可以转换任何仅编码器或仅解码器的转换器模型.

Pegasus is a seq2seq model, you can't directly convert a seq2seq model (encoder-decoder model) using this method. The guide is for BERT which is an encoder model. Any only encoder or only decoder transformer model can be converted using this method.

要转换 seq2seq 模型(编码器-解码器)，您必须将它们拆分并分别转换，将编码器转换为 onnx，将解码器转换为 onnx.您可以按照本指南T5 也是一个 seq2seq 模型)

To convert a seq2seq model (encoder-decoder) you have to split them and convert them separately, an encoder to onnx and a decoder to onnx. you can follow this guide (it was done for T5 which is also a seq2seq model)

为什么会出现此错误?

同时将 PyTorch 转换为 onnx

_ = torch.onnx._export(
                        model,
                        dummy_input,
                        ...
                       )

您需要分别向编码器和解码器提供一个虚拟变量.默认情况下，使用此方法进行转换时，它会为编码器提供虚拟变量.由于这种转换方法不接受这个 seq2seq 模型的解码器，它不会给解码器一个虚拟变量，你会得到上面的错误.ValueError:您必须指定decoder_input_ids 或decoder_inputs_embeds

you need to provide a dummy variable to both encoder and to the decoder separately. by default when converting using this method it provides the encoder the dummy variable. Since this method of conversion didn't accept decoder of this seq2seq model, it won't give a dummy variable to the decoder and you get the above error. ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

这篇关于如何将 HuggingFace 的 Seq2seq 模型转换为 onnx 格式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！