将像对象这样的文件保存到 s3 我得到错误:Unicode 对象必须在散列之前编码 [英] Saving file like object to s3 i get error: Unicode-objects must be encoded before hashing

问题描述

这是我的代码.

import boto3
import pandas as pd
import requests
from io import StringIO

campaign_buffer=StringIO()

r = requests.get('https://.... output=csv....')

if r.status_code==200:
    r.encoding='utf-8'
    request_txt = r.text
    campaigns = StringIO(request_txt)
    campaigns_pd = pd.read_csv(campaigns, sep=",")
    campaigns_pd.columns=campaigns_pd.columns.str.replace(':','_')
    campaigns_pd.drop('images_thumb', inplace=True, axis=1)
    campaigns_pd.to_csv(campaign_buffer)
else:
    print('error')

bucket = 'name'
key = 'folder/test.csv'

client = boto3.client('s3')
client.upload_fileobj(campaign_buffer, bucket, key)

最后一行代码导致错误:类型错误:在散列之前必须对 Unicode 对象进行编码

Last line of code caused error: TypeError: Unicode-objects must be encoded before hashing

有什么解决问题的想法吗?

Any ideas how to solve the problem?

推荐答案

你正在写一个 StringIO()，它没有内在编码，你不能写一些可以't 被编码成字节到 S3 中.要做到这一点，而不必重新编码您写入 campaing_buffer 的任何内容:

You're writing to a StringIO(), which has no intrinsic encoding, and you can't write something that can't be encoded into bytes into S3. To do this without having to re-encode whatever you've written to campaing_buffer:

1. 使您的 campaign_buffer 成为 BytesIO() 而不是 StringIO()
2. 将 mode=wb" 和 encoding=UTF-8" 添加到 to_csv 调用
3. 在上传之前执行 campaign_buffer.seek(0) 来回退内存中的文件

1. Make your campaign_buffer a BytesIO() instead of a StringIO()
2. Add mode="wb" and encoding="UTF-8" to the to_csv call
3. Do campaign_buffer.seek(0) to rewind the in-memory file before uploading

这篇关于将像对象这样的文件保存到 s3 我得到错误:Unicode 对象必须在散列之前编码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！