如何修复“在解释器中至少有 1 个以 numpy 数组或切片的形式引用内部数据"并在 tf.lite 上运行推理 [英] how to fix "There is at least 1 reference to internal data in the interpreter in the form of a numpy array or slice" and run inference on tf.lite

问题描述

我正在尝试在 mnist keras 模型上使用 tf.lite 运行推理，我通过根据 这个

I'm trying to run inference using tf.lite on an mnist keras model that I optimized by doing post-training-quantization according to this

RuntimeError: There is at least 1 reference to internal data
in the interpreter in the form of a numpy array or slice. Be sure to
only hold the function returned from tensor() if you are using raw
data access.

它发生在我将图像调整为 4 维或解释器本身(如注释行所示)之后；因为在此之前的错误类似于预期有 4 个维度，但发现有 3 个".代码如下:

It happens after I resize either the images to be in 4 dimensions, or the interpreter itself as seen in the commented line; since the error before this was something like "expected 4 dimensions but found 3". Here is the code:

import tensorflow as tf
tf.enable_eager_execution()
import numpy as np
from tensorflow.keras.datasets import mnist
import matplotlib.pyplot as plt
%matplotlib inline

mnist_train, mnist_test = tf.keras.datasets.mnist.load_data()
images, labels = tf.cast(mnist_test[0], tf.float32)/255.0, mnist_test[1]
images = np.reshape(images,[images.shape[0],images.shape[1],images.shape[2],1])
mnist_ds = tf.data.Dataset.from_tensor_slices((images, labels)).batch(1, drop_remainder = True)

interpreter = tf.lite.Interpreter(model_path="C:\\Users\\USER\\Documents\\python\\converted_quant_model_cnn_5_100.tflite")
#tf.lite.Interpreter.resize_tensor_input(interpreter, input_index="index" , tensor_size=([1,28,28,1]) )

interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]

for img, label in mnist_ds.take(1):
  break
#print(img.get_shape)
interpreter.set_tensor(input_index, img)
interpreter.invoke()
predictions = interpreter.get_tensor(output_index)

推荐答案

我在 tflite 模型上运行推理时遇到了同样的问题.回溯时，我最终读取了发生此运行时错误的函数.

I was facing the same issue while running inference on a tflite model. When traced back, I ended up reading the function in which this runtime error occurs.

导致此错误的函数是:

def _ensure_safe(self)

和

def _safe_to_run(self)

函数_safe_to_run()"是从函数_ensure_safe()"内部调用的._safe_to_run() 函数要么返回 True 要么返回 False.当它返回 False 时，会发生上述运行时错误.

The function "_safe_to_run()" is called from within the function "_ensure_safe()". _safe_to_run() function either returns True of False. When it return False the above runtime error occurs.

当存在 numpy 数组缓冲区时返回 False.这意味着运行可能会破坏(或更改)内部分配内存的 tflite 调用是不安全的.

It returns False when there exist numpy array buffers. This means it is not safe to run tflite calls that may destroy (or alter) internally allocated memory.

因此，要使_ensure_safe()"函数不引发此运行时错误，我们必须确保没有指向内部缓冲区的 numpy 数组处于活动状态.

So for "_ensure_safe()" function to not raise this runtime error we have to make sure that no numpy arrays pointing to internal buffers are active.

此外，为了更清楚，请注意函数_ensure_safe()"应该从任何将调用 _interpreter 上可能重新分配内存的函数的函数调用.因此当你调用函数

Also, for more clarity note that the function "_ensure_safe()" should be called from any function that will call a function on _interpreter that may reallocate memory. Thus when you call the function

interpreter.allocate_tensors()

正如你在上面的代码中提到的，这个interpreter.allocate_tensors()"函数在内部做的第一件事是调用_ensure_safe()"函数，因为interpreter.allocate_tensors()"涉及改变内部分配的内存(在这种情况下，正如名称所暗示的那样，更改意味着分配")._ensure_safe()"也被调用的另一个例子是invoke()"函数被调用时.并且有很多这样的功能，但你懂的.

as you have mentioned in the code above, the first thing that this "interpreter.allocate_tensors()" function does internally is call the "_ensure_safe()" funciton as the "interpreter.allocate_tensors()" involves altering the internal allocated memory (in this case altering means "allocating" as the name suggests). The other example where "_ensure_safe()" is also called is when "invoke()" function is called. And there are many such functions, but you get the idea.

既然知道根本原因和工作，为了克服这个运行时错误，即没有指向内部缓冲区的 numpy 数组，我们必须清除它们.

Now that the root cause and working is known, to overcome this runtime error i.e to have no numpy arrays pointing to internal buffers, we have to clear them.

清除它们:

a).关闭您的 jupyter notebook 并重新启动内核，因为这将清除所有 numpy 数组/切片

a). Either shutdown you jupyter notebook and restart the kernel, as this will clear all numpy arrays/slices

b).或者简单地再次加载模型，即在你的 jupyter notebook 中再次运行这一行:

b). Or simply load the model again i.e run this line again in you jupyter notebook:

interpreter = tf.lite.Interpreter(model_path="C:\\Users\\USER\\Documents\\python\\converted_quant_model_cnn_5_100.tflite")

这有望解决您的问题，我向您保证它对我有用.

This hopefully solves your problem, I assure you it did for me.

如果这两个选项都没有，那么在上面的解释中我已经指出了为什么"会发生这个错误.因此，如果您发现没有指向内部缓冲区的 numpy 数组"的其他方法，请分享.

If both of these options does not, then in the above explanation I have pointed out "why" this error occurs. So if you find out other ways of "having no numpy arrays pointing to internal buffers", do share.

参考:https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/python/interpreter.py

这篇关于如何修复“在解释器中至少有 1 个以 numpy 数组或切片的形式引用内部数据"并在 tf.lite 上运行推理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！