任何解决方案，以获取列表到numpy数组以用于keras输入数据? [英] Any workaround to getting a list to numpy array for keras input data?

问题描述

我正在尝试在keras上训练CNN模型，我的数据看起来像这样

I'm trying to train a CNN model on keras, my data looks like this

type(datain)
<class 'list'>
len(datain)
35000
type(datain[0])
<class 'numpy.ndarray'>
datain[0].shape
(256,256,1)

作为输入数据的数组列表，当我尝试训练网络时遇到此错误

And being my input data a list of arrays I get this error when trying to train the network

AttributeError: 'list' object has no attribute 'shape'

，但是尝试执行此处建议的np.array(datain)之类的操作时 https://github.com/keras-team/keras/issues/4823 我的计算机死机/崩溃.使用python list定义我的输入总共需要60秒钟，但是如果我从一开始就尝试使用numpy数组，但是每个(256,256,1)数组需要1 sec，并且如果我打算进行各种测试和修改的话，这将花费太多时间我的网络，
有什么办法可以解决这个问题?
用任何方式为喀拉拉邦使用列表?
定义numpy数组的另一种方法?
还是我误会了什么?

but when trying to do something like np.array(datain) as suggested here https://github.com/keras-team/keras/issues/4823 my computer freezes/crashes. defining my input using python list takes like 60 seconds in total, but if I try as numpy arrays from the beginning but takes like 1 sec per (256,256,1) array, and is way too much time if I intent of doing various test and modifications to my network,
is there any work around for this problem?
any way to use lists for keras?
a different way to define a numpy array?
or am I misunderstanding something?

推荐答案

根据数据创建生成器.

A generator是python概念，它循环并产生结果.对于Keras，您的生成器应该无限期地生成X_train和y_train批次.

A generator is a python concept, it loops and yields results. For Keras, your generator should yield batches of X_train and y_train indefinitely.

因此，您可以创建一个简单的生成器:

So, a simple generator that you can make is:

def generator(batch_size,from_list_x,from_list_y):

    assert len(from_list_x) == len(from_list_y)
    total_size = len(from_list_x)

    while True #keras generators should be infinite

        for i in range(0,total_size,batch_size):
            yield np.array(from_list_x[i:i+batch_size]), np.array(from_list_y[i:i+batch_size])

在训练中使用发生器:

model.fit_generator(generator(size,datain,dataout),
                    steps_per_epoch=len(datain)//size, 
                    epochs=...,...)

这篇关于任何解决方案，以获取列表到numpy数组以用于keras输入数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！