如何将数据从Google Cloud Storage导入到Google Colab [英] How to import data from Google Cloud Storage to Google Colab

问题描述

当前，我正在处理10 GB的数据集.我已经将其上传到了Google云存储中，但是我不知道如何将其导入到Google Colab中.

Currently I am working on a data set that is of 10 GB. I have uploaded it on google cloud storage but I don't know how to import it in google colab.

推荐答案

from google.colab import auth
auth.authenticate_user()

运行此命令后，将生成一个链接，您可以单击它并完成登录.

Once you run this, a link will be generated, you can click on it and get the signing in done.

!echo "deb http://packages.cloud.google.com/apt gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
!apt -qq update
!apt -qq install gcsfuse

使用它在colab上安装gcsfuse. Cloud Storage FUSE是一种开源FUSE适配器，可用于将Cloud Storage存储桶作为文件系统挂载在Colab，Linux或macOS系统上.

Use this to install gcsfuse on colab. Cloud Storage FUSE is an open source FUSE adapter that allows you to mount Cloud Storage buckets as file systems on Colab, Linux or macOS systems.

!mkdir folderOnColab
!gcsfuse folderOnBucket/content/ folderOnColab

使用它来挂载目录. (folderOnBucket是不带gs://部分的GCS存储桶URL)

Use this to mount the directories. (folderOnBucket is the GCS bucket URL without the gs:// part)

您可以使用此文档进行进一步阅读. https://cloud.google.com/storage/docs/gcs-fuse

You can use this docs for further reading. https://cloud.google.com/storage/docs/gcs-fuse

这篇关于如何将数据从Google Cloud Storage导入到Google Colab的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！