无法将 NumPy 数组转换为张量(不支持的对象类型字典) [英] Failed to convert a NumPy array to a Tensor (Unsupported object type dict)

问题描述

我的方法我认为它的问题是

my method i thought that the problem from it is

 history = model.fit_generator(train_generator, epochs=epochs, steps_per_epoch=train_steps, verbose=1, callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)

def data_generator(descriptions, photos, tokenizer, max_length, imgsIds, vocab_size):
    while 1:
        for ind in range(len(imgsIds)):
            photo = photos[ind]
            key = imgsIds[ind]
            desc_list = descriptions[str(key)]
            in_img, in_seq, out_word = create_sequences(
                tokenizer, max_length, desc_list, photo, vocab_size)
            yield [in_img, in_seq], out_word

我得到了

Failed to convert a NumPy array to a Tensor (Unsupported object type dict).

如果有什么我应该添加的请评论..谢谢

if there is anything i should add it please comment .. Thanks

Traceback (most recent call last):
  File "fit.py", line 271, in <module>
    main(sys.argv)
  File "fit.py", line 268, in main
    fit_model(train, train_descriptions, train_rnn_input, val, val_descriptions, val_rnn_input)
  File "fit.py", line 255, in fit_model
    history = model.fit_generator(train_generator, epochs=epochs, steps_per_epoch=train_steps, verbose=1, callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1479, in fit_generator
    initial_epoch=initial_epoch)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 872, in fit
    return_dict=True)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1057, in evaluate
    model=self)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1112, in __init__
    model=model)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 775, in __init__
    peek = _process_tensorlike(peek)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1013, in _process_tensorlike
    inputs = nest.map_structure(_convert_numpy_and_scipy, inputs)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 617, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 617, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1008, in _convert_numpy_and_scipy
    return ops.convert_to_tensor(x, dtype=dtype)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1341, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_conversion_registry.py", line 52, in _default_conversion_function
    return constant_op.constant(value, dtype, name=name)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 262, in constant
    allow_broadcast=True)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 270, in _constant_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/path/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 96, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type dict).
2021-06-27 04:46:22.936001: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
     [[{{node PyFunc}}]]

编辑

def create_sequences(tokenizer, max_length, desc_list, photo, vocab_size):
    X1, X2, y = list(), list(), list()
    for desc in desc_list:
        seq = tokenizer.texts_to_sequences([desc])[0]
        for i in range(1, len(seq)):
            in_seq, out_seq = seq[:i], seq[i]
            in_seq = pad_sequences([in_seq], maxlen=max_length)[0]
            out_seq = to_categorical([out_seq], num_classes=vocab_size)[0]
            X1.append(np.array(photo).astype(np.float32))
            X2.append(np.array(in_seq).astype(np.float32))
            y.append(np.array(out_seq).astype(np.float32))
    return array(X1), array(X2), array(y)

推荐答案

此错误表示数据中的某些值或所有值没有有效的数据类型可以转换.

原因:

此错误的常见原因是数组的值在图形模式下不是给定的 dtype.可能是因为某些值是 NaN 或 None 或者所有值都是不支持转换为张量的格式，例如 python 字典.

Common reason for this error is that values of the array are not of given dtype in graph mode. It may be because some values are NaN or None or all values are in a format which is not supported to convert to a tensor, such as a python dictionary.

解决方案:

这个问题可以通过将数据转换为预期的 dtype 来解决，例如将 x=np.asarray(x).astype(np.float32) 等方法应用于之前的输入数据给模型喂食.它还支持 NaN 值问题.请注意，最好对 None 数据值进行一些预处理，并将它们替换为 DataFrame.fillna().

This issue would be fixed by converting the data to the expected dtype, for example applying methods such as x=np.asarray(x).astype(np.float32) to the input data before feeding to the model. It also supports the NaN values issue. Note that it's better to do some preprocessing to None data values, and replace them with methods like DataFrame.fillna().

但是在您的数据类型不受支持的情况下(例如 python 字典)，您无法使用上述方法解决问题.您应该更改数据结构，然后将其提供给模型.注意即使 np.asarray 也不能改变结构.你可能会得到数据的类型 numpy 数组，但结构保持不变，不能被网络处理.所以不要把类型作为不是 dict 的证据.下面是一些例子:

But in scenarios where your data type is something unsupported (like python dictionary) you can not solve the problem with the above approach. You should change the data structure then feed it to the model. Note that even np.asarray can not change the structure. You may get the type of data numpy array, but the structure remains the same and can not be handled by network. So don't get the type as an evidence for not being dict. Here is some example:

x = {1:1,2:2,3:3,4:4,5:5}
print(type(x))      #<class 'dict'>
x = np.asarray(x)
print(type(x))      #<class 'numpy.ndarray'> #the type is changed
print(x)            #{1: 1, 2: 2, 3: 3, 4: 4, 5: 5} #the structure has not been changed
    

---

您的场景:

正如您的代码所反映的那样，您将数据转换为 numpy 数组和浮点数.所以，即使你有 None 变量，你也不会出错.因此，正如错误所反映的(Unsupported object type dict)，您的输入变量之一([in_img, in_seq], out_word)是一个字典.根据您的代码，in_seq 和 out_seq 是列表.所以，它应该是从 photo 变量启动的 in_img.所以，检查这个变量数据.它很可能拥有像数据一样的字典.不要关注类型(print(type(photo)))，因为如我上面的代码，它可能是一个numpy.ndarray，但持有一个字典数据.

As your code reflects, you converted the data to the numpy array and float. So, even you have None variables, you won't get error. So, as the error reflects (Unsupported object type dict), one of your input variables ([in_img, in_seq], out_word) is a dictionary. Based on your code, in_seq and out_seq are lists. So, it should be in_img which initiate from photo variable. So, check this variable data. It is likely holds a dictionary like data. Don't pay attention to the type (print(type(photo))), because as my above code, it could be a numpy.ndarray, but hold a dictionary data.

这篇关于无法将 NumPy 数组转换为张量(不支持的对象类型字典)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！