请求 - 如何流式上传 - 部分文件 [英] requests - how to stream upload - partial file

问题描述

我的目标是使用请求对文件的一部分执行 PUT 并流式传输文件(即，不将其加载到内存中然后执行 PUT).

My goal is to do a PUT of part of a file using requests and stream the file (i.e., not load it into memory and then do the PUT).

此页面说明了如何对整个文件执行此操作:

This page explains how you would do that for an entire file:

Requests 支持流式上传，这允许您发送大流或文件而不将它们读入内存.流和上传，只需为您的身体提供一个类似文件的对象:

Requests supports streaming uploads, which allow you to send large streams or files without reading them into memory. To stream and upload, simply provide a file-like object for your body:

with open('massive-body', 'rb') as f:
    requests.post('http://some.url/streamed', data=f)

但是在我的情况下，我只想发送文件的一个块.有没有办法做到这一点?

However in my case I want to only send one chunk of the file. Is there a way to accomplish this?

在概念上，类似于:

with open('massive-body', 'rb') as f:
    requests.post('http://some.url/streamed', data=f.read(chunksize))

推荐答案

根据 Greg 对我的问题的回答，我认为以下方法最有效:

Based off Greg's answers to my questions I think the following will work best:

首先，您需要一些东西来包装打开的文件，以限制可以读取的数据量:

First you'll need something to wrap your open file so that it limits how much data can be read:

class FileLimiter(object):
    def __init__(self, file_obj, read_limit):
        self.read_limit = read_limit
        self.amount_seen = 0
        self.file_obj = file_obj

        # So that requests doesn't try to chunk the upload but will instead stream it:
        self.len = read_limit

    def read(self, amount=-1):
        if self.amount_seen >= self.read_limit:
            return b''
        remaining_amount = self.read_limit - self.amount_seen
        data = self.file_obj.read(min(amount, remaining_amount))
        self.amount_seen += len(data)
        return data

这应该大致可以作为一个很好的包装对象.然后你会像这样使用它:

This should roughly work as a good wrapper object. Then you would use it like so:

 with open('my_large_file', 'rb') as file_obj:
     file_obj.seek(my_offset)
     upload = FileLimiter(file_obj, my_chunk_limit)
     r = requests.post(url, data=upload, headers={'Content-Type': 'application/octet-stream'})

标头显然是可选的，但是当将数据流式传输到服务器时，作为一个体贴的用户并告诉服务器您发送的内容类型是一个好主意.

The headers are obviously optional, but when streaming data to a server, it's a good idea to be a considerate user and tell the server what the type of the content is that you're sending.

这篇关于请求 - 如何流式上传 - 部分文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！