将LSTM/GRU添加到keras张量流中的BERT嵌入中 [英] add LSTM/GRU to BERT embeddings in keras tensorflow

问题描述

我正在按照以下代码 https:进行BERT嵌入. //github.com/strongio/keras-bert/blob/master/keras-bert.py

这些是代码的重要位(第265-267行):

These are the important bits of the code (lines 265-267):

bert_output = BertLayer(n_fine_tune_layers=3)(bert_inputs)
dense = tf.keras.layers.Dense(256, activation="relu")(bert_output)
pred = tf.keras.layers.Dense(1, activation="sigmoid")(dense)

我想在BertLayer和Dense层之间添加一个GRU

I want to add a GRU between BertLayer and the Dense layer

bert_output = BertLayer(n_fine_tune_layers=3)(bert_inputs)
gru_out = tf.keras.layers.GRU(100, activation='sigmoid')(bert_output)
dense = tf.keras.layers.Dense(256, activation="relu")(gru_out)
pred = tf.keras.layers.Dense(1, activation="sigmoid")(dense)

但出现此错误TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'.

我不确定如何解决此问题.我需要重塑bert_output还是需要创建GRU可以处理的Embedding层?

I am not entirely sure how to address this problem. Do I need to reshape bert_output or do I need to create an Embedding layer that the GRU can handle?

推荐答案

我遇到了相同的错误，对此的解决方法是

I have had the same error, the solution to this is

embedding_size = 768

bert_output = BertLayer(n_fine_tune_layers=3)(bert_inputs)
# Reshape bert_output before passing it the GRU
bert_output_ = tf.keras.layers.Reshape((max_seq_length, embedding_size))(bert_output)

gru_out = tf.keras.layers.GRU(100, activation='sigmoid')(bert_output_)
dense = tf.keras.layers.Dense(256, activation="relu")(gru_out)
pred = tf.keras.layers.Dense(1, activation="sigmoid")(dense)

我希望它能起作用，您可以参考我的问题(如果需要)

I hope it works, you can refer to my question if needed

这篇关于将LSTM/GRU添加到keras张量流中的BERT嵌入中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！