DataGenerator在Google云端硬盘中找不到文件 [英] DataGenerator does not find the Files in Google Drive

问题描述

在此处使用代码 https://keras.io/api/utils/ python_utils /＃sequence-class ，我编写了一个自定义DataGenerator。

Using the code here https://keras.io/api/utils/python_utils/#sequence-class, I coded a custom DataGenerator.

# Here, `x_set` is list of path to the images
 # and `y_set` are the associated classes.

class DataGenerator(Sequence):
def __init__(self, x_set, y_set, batch_size):
    self.x, self.y = x_set, y_set
    self.batch_size = batch_size

def __len__(self):
    return math.ceil(len(self.x) / self.batch_size)

def __getitem__(self, idx):
    batch_x = self.x[idx * self.batch_size:(idx + 1) *
    self.batch_size]
    batch_y = self.y[idx * self.batch_size:(idx + 1) *
    self.batch_size]

    return np.array([
        resize(imread(file_name), (224, 224))
           for file_name in batch_x]), np.array(batch_y)

我有 X_train 和 X_val ，它们是包含我的图像文件的图像路径以及y_train和y_val的列表，它们是一个热编码标签，并将此数据应用于 DataGenerator 。

I have X_train and X_val, which are lists containing the image paths to my image files and y_train and y_valwhich are one hot encoded labels and applied this data to the DataGenerator.

training_generator = DataGenerator(X_train, y_train, batch_size=32)
validation_generator = DataGenerator(X_val, y_val, batch_size=32)

然后拟合模型：

model.fit_generator(generator=training_generator,
                    validation_data=validation_generator,
                    steps_per_epoch = num_train_samples // batch_size,
                    validation_steps = num_val_samples // batch_size,
                    epochs = 10,
                    use_multiprocessing=True,
                    workers=6)

运行此代码时，出现此错误：

When running this code, I got this error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-54-411e62536182> in <module>()
      5                     epochs = 10,
      6                     use_multiprocessing=True,
----> 7                     workers=6)

16 frames
/usr/local/lib/python3.6/dist-packages/imageio/core/request.py in _parse_uri(self, uri)
    271                 # Reading: check that the file exists (but is allowed a dir)
    272                 if not os.path.exists(fn):
--> 273                     raise FileNotFoundError("No such file: '%s'" % fn)
    274             else:
    275                 # Writing: check that the directory to write to does exist

FileNotFoundError: No such file: '/content/gdrive/My Drive/data/2017-IWT4S-CarsReId_LP-dataset/s01_l01/1_1.png'

X_train 是一个列表，其中以字符串形式包含我在Google云端硬盘中数据的绝对文件路径。

X_train is a list which contains the absolute file path to my data in the Google Drive as a string.

X_train[0]
'/content/gdrive/My Drive/data/2017-IWT4S-CarsReId_LP-dataset/s01_l01/1_1.png'

如何修改代码以使生成器找到我的文件？
那可能是因为Colab Notebook文件和数据不在同一文件夹中吗？ ->我将Colab Jupyter笔记本放在 / content / gdrive /我的驱动器文件夹中，并且我也收到FileNotFoundError。
还是必须将路由路由到正确的文件夹？

How do I have to modify the code in order to make the generator find my files? Is that maybe because the Colab Notebook file and the data are not in the same folder? -> I pu my Colab Jupyter Notebook in the folder "/content/gdrive/My Drive" and I also got the FileNotFoundError". Or do I have to route anyhow to the correct folder?

推荐答案

这是由于Google Drive TimeOut引起的，我通过按照以下建议在控制台中编写来解决此问题：FileNotFoundError：无此类文件：->由于Google Drive超时而发生错误？

This happened due to a Google Drive TimeOut. I solved this issue by writing in my console as recommended here: FileNotFoundError: No such file: -> Error occuring due to TimeOut of Google Drive?

function ConnectButton(){
    console.log("Connect pushed"); 
    document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click() 
}
setInterval(ConnectButton,600);

这篇关于DataGenerator在Google云端硬盘中找不到文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！