使用 sagemaker 将数据上传到 S3 [英] upload data to S3 with sagemaker

问题描述

当我尝试将数据上传到 S3 存储桶时，SageMaker 出现问题.我收到此错误:

<块引用><小时>

NameError Traceback(最近一次调用最后一次)<ipython-input-26-d21b1cb0fcab>在 <module>()19 下载('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')20--->21 upload_to_s3('train', 'caltech-256-60-train.rec')<ipython-input-26-d21b1cb0fcab>在upload_to_s3(频道，文件)中13 数据 = 打开(文件，rb")14 键 = 频道 + '/' + 文件--->15 s3.Bucket(bucket).put_object(Key=key, Body=data)1617NameError:未定义名称bucket"

这是脚本:

import os导入 urllib.request导入 boto3定义下载(网址):文件名 = url.split("/")[-1]如果不是 os.path.exists(filename):urllib.request.urlretrieve(url, 文件名)def upload_to_s3(频道，文件):s3 = boto3.resource('s3')数据 = 打开(文件，rb")键 = 频道 + '/' + 文件s3.Bucket(bucket).put_object(Key=key, Body=data)# caltech-256 下载('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')upload_to_s3('train', 'caltech-256-60-train.rec')

解决方案

正如错误所说，变量 bucket 未定义.你可能想做一些像

bucket = 

在你打电话之前

s3.Bucket(bucket).put_object(Key=key, Body=data)

I have a problem with SageMaker when I try to upload Data into S3 bucket . I get this error :

---

NameError                                 Traceback (most recent call last)
<ipython-input-26-d21b1cb0fcab> in <module>()
     19 download('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')
     20 
---> 21 upload_to_s3('train', 'caltech-256-60-train.rec')

<ipython-input-26-d21b1cb0fcab> in upload_to_s3(channel, file)
     13     data = open(file, "rb")
     14     key = channel + '/' + file
---> 15     s3.Bucket(bucket).put_object(Key=key, Body=data)
     16 
     17 

NameError: name 'bucket' is not defined

Here is the script:

import os
import urllib.request
import boto3

def download(url):
    filename = url.split("/")[-1]
    if not os.path.exists(filename):
        urllib.request.urlretrieve(url, filename)


def upload_to_s3(channel, file):
    s3 = boto3.resource('s3')
    data = open(file, "rb")
    key = channel + '/' + file
    s3.Bucket(bucket).put_object(Key=key, Body=data)


# caltech-256 download('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')

upload_to_s3('train', 'caltech-256-60-train.rec')

解决方案

It is exactly as the error say, the variable bucket is not defined. you might want to do something like

bucket = <name of already created bucket in s3>

before you call

s3.Bucket(bucket).put_object(Key=key, Body=data)

这篇关于使用 sagemaker 将数据上传到 S3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！