如何从本地驱动器上传和保存大量数据到Google Colaboratory？ [英] How to upload and save large data to Google Colaboratory from local drive?

问题描述

我已经从这个Kaggle链接下载了大量的图像训练数据。

https://www.kaggle.com/c/yelp-restaurant-photo-classification/data

如何有效实现以下目标？

1. 在Google Colaboratory中创建项目文件夹


2. 将zip文件上传到项目文件夹


3. 解压缩文件

谢谢

>

编辑：我尝试了下面的代码，但它的崩溃为我的大型zip文件。有没有更好的/有效的方式来做到这一点，我可以指定文件在本地驱动器的位置？

  from google .colab为uploaded.keys（）中的fn导入文件
 uploaded = files.upload（）
 
（
 print（'用户上传文件{name}长度} bytes.format（
 name = fn，length = len（已上传[fn]）））
  

解决方案

 ！pip install kaggle 
 api_token = {username：USERNAME，key：API_KEY } 
导入json 
导入zipfile 
导入os 
以打开（'/ content / .kaggle / kaggle.json'，'w'）作为文件：
 json .dump（api_token，file）
！chmod 600 /content/.kaggle/kaggle.json 
！kaggle配置集-n路径-v /内容
！kaggle比赛下载-c jigsaw- 
 os.chdir（'/ content / competitions / jigsaw-toxic-comment-classification-challenge'）
用于os.listdir（）中的文件：
 zip_ref = zipfile.ZipF ile（file，'r'）
 zip_ref.extractall（）
 zip_ref.close（）
  

第9行稍有改动，没有遇到错误。
来源： https://gist.github.com/jayspeidell/d10b84b8d3da52df723beacc5b15cb27
无法添加为评论原因代表。

I have downloaded large image training data as zip from this Kaggle link

https://www.kaggle.com/c/yelp-restaurant-photo-classification/data

How do I efficiently achieve the following?

1. Create a project folder in Google Colaboratory
2. Upload zip file to project folder
3. unzip the files

Thanks

EDIT: I tried the below code but its crashing for my large zip file. Is there a better/efficient way to do this where I can just specify the location of the file in local drive?

from google.colab import files
uploaded = files.upload()

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))

解决方案

!pip install kaggle
api_token = {"username":"USERNAME","key":"API_KEY"}
import json
import zipfile
import os
with open('/content/.kaggle/kaggle.json', 'w') as file:
    json.dump(api_token, file)
!chmod 600 /content/.kaggle/kaggle.json
!kaggle config set -n path -v /content
!kaggle competitions download -c jigsaw-toxic-comment-classification-challenge
os.chdir('/content/competitions/jigsaw-toxic-comment-classification-challenge')
for file in os.listdir():
    zip_ref = zipfile.ZipFile(file, 'r')
    zip_ref.extractall()
    zip_ref.close()

There is minor change on line 9, without which was encountering error. source: https://gist.github.com/jayspeidell/d10b84b8d3da52df723beacc5b15cb27 couldn't add as comment cause rep.

这篇关于如何从本地驱动器上传和保存大量数据到Google Colaboratory？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！