从.hdf5中的权重和.json中的选项实例化keras中的ELMo模型 [英] Instantiate ELMo model in keras from weights in .hdf5 and options in .json

问题描述

在 https://allennlp.org/elmo 中提供了经过预训练的ELMo模型.. >

我将如何使用提供的文件?

我认为我必须从json文件重构模型，然后将.hdf5文件的权重加载到模型中. 但是json格式似乎不适用于keras.models.model_from_json.我得到了错误: ValueError: Improper config format: ...

解决方案

使用tensorflow_hub加载ELMo模型，例如:

import tensorflow as tf
import tensorflow_hub as hub
from keras.layers import Lambda
from keras.models import Input
from keras import backend as K
sess = tf.Session()
K.set_session(sess)

batch_size = 64
max_len = 100
elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())

def ElmoEmbedding(x):
    return elmo_model(inputs={"tokens": tf.squeeze(tf.cast(x,tf.string)),
                              "sequence_len": tf.constant(batch_size*[max_len])},
                      signature="tokens",
                      as_dict=True)["elmo"]
input_x = Input(shape=(max_len,), dtype=tf.string)
embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 128))(input_x)

The pre-trained ELMo models are provided at https://allennlp.org/elmo.

How would I use the files provided?

I think that I have to reconstruct the model from the json file, and then load the weights from .hdf5 file into the model. But json format doesn't seem to work for keras.models.model_from_json. I got the error: ValueError: Improper config format: ...

解决方案

Using tensorflow_hub to load ELMo model, an example:

import tensorflow as tf
import tensorflow_hub as hub
from keras.layers import Lambda
from keras.models import Input
from keras import backend as K
sess = tf.Session()
K.set_session(sess)

batch_size = 64
max_len = 100
elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())

def ElmoEmbedding(x):
    return elmo_model(inputs={"tokens": tf.squeeze(tf.cast(x,tf.string)),
                              "sequence_len": tf.constant(batch_size*[max_len])},
                      signature="tokens",
                      as_dict=True)["elmo"]
input_x = Input(shape=(max_len,), dtype=tf.string)
embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 128))(input_x)

这篇关于从.hdf5中的权重和.json中的选项实例化keras中的ELMo模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！