使用Tensorflow-Hub的ELMo时会大大增加内存消耗 [英] Strongly increasing memory consumption when using ELMo from Tensorflow-Hub

问题描述

我目前正在尝试比较数百万个文档的相似性.对于在CPU上的第一次测试，我将它们每个减少到50个字符，并尝试一次获得10个字符的ELMo嵌入，如下所示:

I am currently trying to compare the similarity of millions of documents. For a first test on a CPU I reduced them to around 50 characters each and try to get the ELMo Embedding for 10 of them at a time like this:

ELMO = "https://tfhub.dev/google/elmo/2"
for row in file:
    split = row.split(";", 1)
    if len(split) > 1:
        text = split[1].replace("\n", "")
            texts.append(text[:50])
    if i == 300:
        break
    if i % 10 == 0:
        elmo = hub.Module(ELMO, trainable=False)
                 executable = elmo(
                 texts,
                 signature="default",
                 as_dict=True)["elmo"]

    vectors = execute(executable)
    texts = []
    i += 1

但是，即使是这个小例子，在大约300个句子(甚至不保存向量)之后，该程序仍会消耗多达12GB的RAM.这是一个已知问题(我发现的其他问题表明存在类似的问题，但还不是那么极端)还是我犯了一个错误?

However, even with this small example, after around 300 sentences (and not even saving the vectors) the program consumes up to 12GB of RAM. Is this a know issue (the other issues I found suggest something similar, but not quite that extreme) or did I make a mistake?

推荐答案

这是针对没有Eager模式的TensorFlow 1.x(否则使用hub.Module可能会遇到更大的问题).

This is for TensorFlow 1.x without Eager mode, I suppose (or else the use of hub.Module would likely hit bigger problems).

在该编程模型中，您需要首先在TensorFlow图中表达您的计算，然后针对每批数据重复执行该图.

In that programming model, you need to first express your computation in a TensorFlow graph, and then execute that graph repeatedly for each batch of data.

• 使用hub.Module()构造模块并将其应用于将输入张量映射到输出张量都是图构建的一部分，并且应该只发生一次.

• Constructing the module with hub.Module() and applying it to map an input tensor to an output tensor are both parts of graph building and should happen only once.

输入数据上的循环应仅调用session.run()来馈入输入并从固定图中获取输出数据.

The loop over the input data should merely call session.run() to feed input and fetch output data from the fixed graph.

幸运的是，已经有一个实用程序功能可以为您完成所有这些操作:

Fortunately, there is already a utility function to do all this for you:

import numpy as np
import tensorflow_hub as hub

# For demo use only. Extend to your actual I/O needs as you see fit.
inputs = (x for x in ["hello world", "quick brown fox"])

with hub.eval_function_for_module("https://tfhub.dev/google/elmo/2") as f:
  for pystr in inputs:
    batch_in = np.array([pystr])
    batch_out = f(batch_in)
    print(pystr, "--->", batch_out[0])

就原始TensorFlow而言，这对您的作用大致是这样:

What this does for you in terms of raw TensorFlow is roughly this:

module = Module(ELMO_OR_WHATEVER)
tensor_in = tf.placeholder(tf.string, shape=[None])  # As befits `module`.
tensor_out = module(tensor_in)

# This kind of session handles init ops for you.
with tf.train.SingularMonitoredSession() as sess:
  for pystr in inputs:
    batch_in = np.array([pystr])
    batch_out = sess.run(tensor_out, feed_dict={tensor_in: batch_in}
    print(pystr, "--->", batch_out[0])

如果您的需求对于with hub.eval_function_for_module ...而言过于复杂，则可以构建更明确的示例.

If your needs are too complex for with hub.eval_function_for_module ..., you could build out this more explicit example.

请注意在循环中既不构造也不调用hub.Module.

Notice how the hub.Module is neither constructed nor called in the loop.

PS:厌倦了担心构建图形还是运行会话的烦恼?然后TF2和热切的执行适合您.查看 https://colab. research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb

PS: Tired of worrying about building graphs vs running sessions? Then TF2 and eager execution are for you. Check out https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb

这篇关于使用Tensorflow-Hub的ELMo时会大大增加内存消耗的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！