如何批量发送包含多个网址的多部分html帖子? [英] How do I batch send a multipart html post with multiple urls?

问题描述

我正在与gmail api对话，并希望对请求进行批处理.他们对此有一个友好的指南， https://developers.google.com/gmail /api/guides/batch ，这表明我应该能够使用multipart/mixed并包含不同的网址.

I am speaking to the gmail api and would like to batch the requests. They have a friendly guide for this here, https://developers.google.com/gmail/api/guides/batch, which suggests that I should be able to use multipart/mixed and include different urls.

我正在使用Python和Requests库，但是不确定如何发出不同的url.像这样的答案如何发送多部分/表单数据" python中有请求吗?没有提及更改该部分的选项.

I am using Python and the Requests library, but am unsure how to issue different urls. Answers like this one How to send a "multipart/form-data" with requests in python? don't mention an option for changing that part.

我该怎么做?

推荐答案

不幸的是，请求在其API中不支持multipart/mixed.几个GitHub问题(#935 和

Unfortunately, requests does not support multipart/mixed in their API. This has been suggested in several GitHub issues (#935 and #1081), but there are no updates on this for now. This also becomes quite clear if you search for "mixed" in the requests sources and get zero results :(

现在，您有几种选择，这取决于要使用Python和3rd-party库的数量.

Now you have several options, depending on how much you want to make use of Python and 3rd-party libraries.

现在，对该问题最明显的答案是使用Google在此处.它带有一个HttpBatchRequest类，该类可以处理所需的批处理请求. 本指南中有详细记录.

Now, the most obvious answer to this problem is to use the official Python API that Google is providing here. It comes with a HttpBatchRequest class that can handle the batch requests that you need. This is documented in detail in this guide.

本质上，您创建一个HttpBatchRequest对象并将所有请求添加到该对象.然后，库会将所有内容放在一起(取自上面的指南):

Essentially, you create an HttpBatchRequest object and add all your requests to it. The library will then put everything together (taken from the guide above):

batch = BatchHttpRequest()
batch.add(service.animals().list(), callback=list_animals)
batch.add(service.farmers().list(), callback=list_farmers)
batch.execute(http=http)

现在，如果由于某种原因您不能或不会使用官方的Google库，则必须自己构建请求主体的一部分.

Now, if for whatever reason you cannot or will not use the official Google libraries you will have to build parts of the request body yourself.

正如我已经提到的，请求不正式支持multipart/mixed.但这并不意味着我们不能强迫"它.创建Request对象时，我们可以使用 files 参数以提供多部分数据.

As I already mentioned, requests does not officially support multipart/mixed. But that does not mean that we cannot "force" it. When creating a Request object, we can use the files parameter to provide multipart data.

files是一个字典，它接受以下格式的四元组值:(文件名，file_object，content_type，标头).文件名可以为空.现在我们需要将Request对象转换为文件(类似)对象.我写了一个小的方法，涵盖了Google示例中的基本示例.部分原因是Google在其Python库中使用的内部方法:

files is a dictionary that accepts 4-tuple values of this format: (filename, file_object, content_type, headers). The filename can be empty. Now we need to convert a Request object into a file(-like) object. I wrote a small method that covers the basic examples from the Google example. It is partly inspired by the internal methods that Google uses in their Python library:

import requests
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart

BASE_URL = 'http://www.googleapis.com/batch'

def serialize_request(request):
    '''Returns the string representation of the request'''
    mime_body = ''

    prepared = request.prepare()

    # write first line (method + uri)
    if request.url.startswith(BASE_URL):
        mime_body = '%s %s\r\n' % (request.method, request.url[len(BASE_URL):])
    else:
        mime_body = '%s %s\r\n' % (request.method, request.url)

    part = MIMENonMultipart('application', 'http')

    # write headers (if possible)
    for key, value in prepared.headers.iteritems():
        mime_body += '%s: %s\r\n' % (key, value)

    if getattr(prepared, 'body', None) is not None:
        mime_body += '\r\n' + prepared.body + '\r\n'

    return mime_body.encode('utf-8').lstrip()

此方法会将requests.Request对象转换为UTF-8编码的字符串，以后可将其用作MIMENonMultipart对象的有效负载，即不同的多部分.

This method will transform a requests.Request object into a UTF-8 encoded string that can later be used a a payload for a MIMENonMultipart object, i.e. the different multiparts.

现在，为了生成实际的批处理请求，我们首先需要将(Google API)请求列表压缩到 requests 库的files词典中.以下方法将获取requests.Request对象的列表，将每个对象转换为MIMENonMultipart，然后返回符合files字典结构的字典:

Now in order to generate the actual batch request, we first need to squeeze a list of (Google API) requests into a files dictionary for the requests lib. The following method will take a list of requests.Request objects, transform each into a MIMENonMultipart and then return a dictionary that complies to the structure of the files dictionary:

import uuid

def prepare_requests(request_list):
    message = MIMEMultipart('mixed')
    output = {}

    # thanks, Google. (Prevents the writing of MIME headers we dont need)
    setattr(message, '_write_headers', lambda self: None)

    for request in request_list:
        message_id = new_id()
        sub_message = MIMENonMultipart('application', 'http')
        sub_message['Content-ID'] = message_id
        del sub_message['MIME-Version']

        sub_message.set_payload(serialize_request(request))

        # remove first line (from ...)
        sub_message = str(sub_message)
        sub_message = sub_message[sub_message.find('\n'):]

        output[message_id] = ('', str(sub_message), 'application/http', {})

    return output

def new_id():
    # I am not sure how these work exactly, so you will have to adapt this code
    return '<item%s:[e-mail]>' % str(uuid.uuid4())[-4:]

最后，我们需要将Content-Type从 multipart/form-data 更改为 multipart/mixed ，并从中删除Content-Disposition和Content-Type标头每个请求部分.这些是由 requests 生成的，不能被files词典覆盖.

Finally, we need to change the Content-Type from multipart/form-data to multipart/mixed and also remove the Content-Disposition and Content-Type headers from each request part. These we generated by requests and cannot be overwritten by the files dictionary.

import re

def finalize_request(prepared):
    # change to multipart/mixed
    old = prepared.headers['Content-Type']
    prepared.headers['Content-Type'] = old.replace('multipart/form-data', 'multipart/mixed')

    # remove headers at the start of each boundary
    prepared.body = re.sub(r'\r\nContent-Disposition: form-data; name=.+\r\nContent-Type: application/http\r\n', '', prepared.body)

我已尽力使用批处理"指南中的Google示例来对此进行测试:

I have tried my best to test this with the Google Example from the Batching guide:

sheep = {
  "animalName": "sheep",
  "animalAge": "5",
  "peltColor": "green"
}

commands = []
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals/pony'))
commands.append(requests.Request('PUT', 'http://www.googleapis.com/batch/farm/v1/animals/sheep', json=sheep, headers={'If-Match': '"etag/sheep"'}))
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals', headers={'If-None-Match': '"etag/animals"'}))

files = prepare_requests(commands)

r = requests.Request('POST', 'http://www.googleapis.com/batch', files=files)
prepared = r.prepare()

finalize_request(prepared)

s = requests.Session()
s.send(prepared)

产生的请求应该与Google在其批处理指南"中提供的请求足够接近:

And the resulting request should be close enough to what Google is providing in their Batching guide:

POST http://www.googleapis.com/batch
Content-Length: 1006
Content-Type: multipart/mixed; boundary=a21beebd15b74be89539b137bbbc7293

--a21beebd15b74be89539b137bbbc7293

Content-Type: application/http
Content-ID: <item8065:[e-mail]>

GET /farm/v1/animals
If-None-Match: "etag/animals"

--a21beebd15b74be89539b137bbbc7293

Content-Type: application/http
Content-ID: <item5158:[e-mail]>

GET /farm/v1/animals/pony

--a21beebd15b74be89539b137bbbc7293

Content-Type: application/http
Content-ID: <item0ec9:[e-mail]>

PUT /farm/v1/animals/sheep
Content-Length: 63
Content-Type: application/json
If-Match: "etag/sheep"

{"animalAge": "5", "animalName": "sheep", "peltColor": "green"}

--a21beebd15b74be89539b137bbbc7293--

最后，我强烈建议您使用Google官方库，但如果您不能使用它，则必须即兴发挥一点:)

In the end, I highly recommend the official Google library but if you cannot use it, you will have to improvise a bit :)

免责声明:我实际上没有尝试将此请求发送到Google API端点，因为身份验证过程太麻烦了.我只是想尽可能地接近批处理"指南中描述的HTTP请求. \ r和\ n行尾可能存在一些问题，具体取决于Google端点的严格程度.

Disclaimer: I havent actually tried to send this request to the Google API Endpoints because the authentication procedure is too much of a hassle. I was just trying to get as close as possible to the HTTP request that is described in the Batching guide. There might be some problems with \r and \n line endings, depending on how strict the Google Endpoints are.

来源:

• 请求github (尤其是问题#935和#1081)
• 请求API文档
• Google Python API

• requests github (especially issues #935 and #1081)
• requests API documentation
• Google APIs for Python

这篇关于如何批量发送包含多个网址的多部分html帖子?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！