如何使用 python 请求 POST JSON/xml 文件的多部分列表 [英] how to POST multipart list of JSON/xml files using python requests

问题描述

在 python2.7 中，我使用 requests 与 REST 端点通信.我可以向它上传单个 JSON 和 xml 对象.为了加快速度，我想使用 multipart 上传多个 json 对象.

我有一个 curl 命令，它显示了它是如何完成的，它是有效的.我需要在 python 请求 POST 命令中翻译它.

工作卷发:

curl --anyauth --user admin:admin -X POST --data-binary @sample-body -i -H "内容类型:多部分/混合；边界=边界" http://localhost:8058/v1/resources/sight-ingest?rs:transform=aireco-transform&rs:title=file1.xml&rs:title=file2.xml&rs:title=file3.xml"

注意事项:我需要发送包含标题"参数列表的自定义参数列表，不能通过传递字典来做到这一点吗?但我们可以解决这个问题.

我的蟒蛇踪迹:

导入请求files = {'file1': ('foo.txt', 'foo
contents
','text/plain'),'file2': ('bar.txt', 'bar 内容', 'text/plain'),'file3': ('baz.txt', 'baz 内容', 'text/plain')}headers = {'Content-Type':'multipart/mixed','Content-Disposition':'attachment','boundary':'GRENS'}params={'title':'file1','title':'file2','title':'file2'}r = requests.Request('POST', 'http://example.com', files=files, headers=headers, params=params)打印 r.prepare().url打印 r.prepare().headers打印 r.prepare().body

给我:

http://example.com/?title=file2{'boundary':'GRENS'，'Content-Type':'multipart/mixed'，'Content-Length':'471'，'Content-Disposition':'attachment'}--7f18a6c1b09f42009228f600b0af35fd内容配置:表单数据；名称=文件3"；文件名=baz.txt"内容类型:文本/纯文本巴兹内容--7f18a6c1b09f42009228f600b0af35fd内容配置:表单数据；名称=文件2"；文件名=bar.txt"内容类型:文本/纯文本酒吧内容--7f18a6c1b09f42009228f600b0af35fd内容配置:表单数据；名称=文件1"；文件名=foo.txt"内容类型:文本/纯文本富内容--7f18a6c1b09f42009228f600b0af35fd--

问题:

• 正文中似乎没有使用标题中的内容?
• 我可以设置自己的边界吗?'GRENS' 不是用在身体上吗?
• 我可以像 curl 示例一样传递标题参数列表(使用相同的键)吗?

解决方案

不要使用字典，使用 (key, value) 元组列表作为查询参数:

params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]

否则你最终只会得到一键.

您应该不设置Content-Type标头；当您使用 files 参数时，requests 将为您正确设置；这样，正确的边界也将被包括在内.你永远不应该自己直接设置边界，真的:

params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]r = requests.post('http://example.com',文件=文件，标题=标题，参数=参数)

您可以通过将第 4 个元素添加到每个文件的元组以获得额外的标题来设置每个文件部分的标题，但在您的情况下，您不应该尝试设置 Content-Disposition 标题自己;无论如何它都会被覆盖.

内省准备好的请求对象然后给你:

<预><代码>>>>进口请求>>>从 pprint 导入 pprint>>>files = {'file1': ('foo.txt', 'foo contents ','text/plain'),... 'file2': ('bar.txt', 'bar contents', 'text/plain'),... 'file3': ('baz.txt', 'baz 内容', 'text/plain')}>>>标题 = {'内容处理':'附件'}>>>params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]>>>r = requests.Request('POST', 'http://example.com',...文件=文件，标题=标题，参数=参数)>>>准备 = r.prepare()>>>准备好的网址'http://example.com/?title=file1&title=file2&title=file3'>>>pprint(dict(prepared.headers)){'内容处置':'附件'，'内容长度':'471'，'内容类型':'多部分/表单数据；边界=7312ccd96db94419bf1d97f2c54bbad1'}>>>打印prepared.body--7312ccd96db94419bf1d97f2c54bbad1内容配置:表单数据；名称=文件3"；文件名=baz.txt"内容类型:文本/纯文本巴兹内容--7312ccd96db94419bf1d97f2c54bbad1内容配置:表单数据；名称=文件2"；文件名=bar.txt"内容类型:文本/纯文本酒吧内容--7312ccd96db94419bf1d97f2c54bbad1内容配置:表单数据；名称=文件1"；文件名=foo.txt"内容类型:文本/纯文本富内容--7312ccd96db94419bf1d97f2c54bbad1--

如果您绝对必须使用 multipart/mixed 而不是 multipart/form-data，则您必须自己构建 POST 主体并从中设置标题.包含的 urllib3 工具 应该能够为您做到这一点:

from requests.packages.urllib3.fields import RequestField从 requests.packages.urllib3.filepost 导入 encode_multipart_formdata字段 = []对于 files.items() 中的名称(文件名、内容、mimetype):rf = RequestField(名称=名称，数据=内容，文件名=文件名)rf.make_multipart(content_disposition='attachment', content_type=mimetype)fields.append(rf)post_body, content_type = encode_multipart_formdata(字段)content_type = ''.join(('multipart/mixed',) + content_type.partition(';')[1:])headers = {'Content-Type': content_type}requests.post('http://example.com', data=post_body, headers=headers, params=params)

或者你可以使用 email 包做同样的事情:

from email.mime.multipart import MIMEMultipart从 email.mime.text 导入 MIMEText正文 = MIMEMultipart()对于 files.items() 中的名称(文件名、内容、mimetype):part = MIMEText(contents, _subtype=mimetype.partition('/')[-1], _charset='utf8')part.add_header('Content-Disposition', 'attachment', filename=filename)body.attach(部分)post_body = body.as_string().partition('

')[-1]content_type = body['内容类型']headers = {'Content-Type': content_type}requests.post('http://example.com', data=post_body, headers=headers, params=params)

但考虑到此方法需要您设置字符集(我假设 JSON 和 XML 为 UTF-8)并且它很可能对内容使用 Base64 编码:

<预><代码>>>>正文 = MIMEMultipart()>>>对于 files.items() 中的名称(文件名、内容、mimetype):... part = MIMEText(contents, _subtype=mimetype.partition('/')[-1], _charset='utf8')... part.add_header('Content-Disposition', 'attachment', filename=filename)... body.attach(部分)...>>>post_body = body.as_string().partition(' ')[-1]>>>content_type = body['内容类型']>>>打印 post_body--================1364782689914852112==MIME 版本:1.0内容类型:文本/纯文本；字符集=utf-8"内容传输编码:base64Content-Disposition:附件；文件名=baz.txt"YmF6IGNvbnRlbnRz--================1364782689914852112==MIME 版本:1.0内容类型:文本/纯文本；字符集=utf-8"内容传输编码:base64Content-Disposition:附件；文件名=bar.txt"YmFyIGNvbnRlbnRz--================1364782689914852112==MIME 版本:1.0内容类型:文本/纯文本；字符集=utf-8"内容传输编码:base64Content-Disposition:附件；文件名=foo.txt"Zm9vCmNvbnRlbnRzCg==--================1364782689914852112==-->>>打印内容类型多部分/混合；边界="================1364782689914852112=="

In python2.7 I use requests to communicate with a REST endpoint. I can upload single JSON and xml objects to it. To speed things up I want to upload multiple json objects using multipart.

I have a curl command that show how it can be done and this works. I need to translate that in a python requests POST command.

WORKING curl POST:

curl --anyauth --user admin:admin -X POST --data-binary @sample-body 
     -i -H "Content-type: multipart/mixed; boundary=BOUNDARY" 
     "http://localhost:8058/v1/resources/sight-ingest?rs:transform=aireco-transform&rs:title=file1.xml&rs:title=file2.xml&rs:title=file3.xml"

Things to note: I need to send custom parameter list including a list of 'title' parameters, cannot do that by passing a dict? but we can potentially work around this.

my python trail:

import requests
files = {'file1': ('foo.txt', 'foo
contents
','text/plain'), 
          'file2': ('bar.txt', 'bar contents', 'text/plain'),
          'file3': ('baz.txt', 'baz contents', 'text/plain')}

headers = {'Content-Type': 'multipart/mixed','Content-Disposition': 'attachment','boundary': 'GRENS'}
params={'title':'file1','title':'file2','title':'file2'}
r = requests.Request('POST', 'http://example.com', files=files , headers=headers, params=params)
print r.prepare().url
print r.prepare().headers
print r.prepare().body

Gives me:

http://example.com/?title=file2
{'boundary': 'GRENS', 'Content-Type': 'multipart/mixed', 'Content-Length': '471', 'Content-Disposition': 'attachment'}
--7f18a6c1b09f42009228f600b0af35fd
Content-Disposition: form-data; name="file3"; filename="baz.txt"
Content-Type: text/plain

baz contents
--7f18a6c1b09f42009228f600b0af35fd
Content-Disposition: form-data; name="file2"; filename="bar.txt"
Content-Type: text/plain

bar contents
--7f18a6c1b09f42009228f600b0af35fd
Content-Disposition: form-data; name="file1"; filename="foo.txt"
Content-Type: text/plain

foo
contents

--7f18a6c1b09f42009228f600b0af35fd--

Questions:

• Seems the stuff in the header is not used in the body?
• Can i set my own boundary? 'GRENS' is not used in the body?
• Can I pass a list of title parameters (with same key) as in curl example?

解决方案

Do not use a dictionary, use a list of (key, value) tuples for your query parameters:

params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]

otherwise you'll end up with just the one key.

You should not set the Content-Type header; requests will set that correctly for you when you use the files parameter; that way the correct boundary will also be included. You should never set the boundary directly yourself, really:

params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]
r = requests.post('http://example.com', 
                  files=files, headers=headers, params=params)

You can set headers per file part by adding a 4th element to the per-file tuple for extra headers, but in your case, you should not try to set the Content-Disposition header yourself; it'll be overwritten anyway.

Introspecting a prepared request object then gives you:

>>> import requests
>>> from pprint import pprint
>>> files = {'file1': ('foo.txt', 'foo
contents
','text/plain'), 
...           'file2': ('bar.txt', 'bar contents', 'text/plain'),
...           'file3': ('baz.txt', 'baz contents', 'text/plain')}
>>> headers = {'Content-Disposition': 'attachment'}
>>> params = [('title', 'file1'), ('title', 'file2'), ('title', 'file3')]
>>> r = requests.Request('POST', 'http://example.com',
...                      files=files, headers=headers, params=params)
>>> prepared = r.prepare()
>>> prepared.url
'http://example.com/?title=file1&title=file2&title=file3'
>>> pprint(dict(prepared.headers))
{'Content-Disposition': 'attachment',
 'Content-Length': '471',
 'Content-Type': 'multipart/form-data; boundary=7312ccd96db94419bf1d97f2c54bbad1'}
>>> print prepared.body
--7312ccd96db94419bf1d97f2c54bbad1
Content-Disposition: form-data; name="file3"; filename="baz.txt"
Content-Type: text/plain

baz contents
--7312ccd96db94419bf1d97f2c54bbad1
Content-Disposition: form-data; name="file2"; filename="bar.txt"
Content-Type: text/plain

bar contents
--7312ccd96db94419bf1d97f2c54bbad1
Content-Disposition: form-data; name="file1"; filename="foo.txt"
Content-Type: text/plain

foo
contents

--7312ccd96db94419bf1d97f2c54bbad1--

If you absolutely must have multipart/mixed and not multipart/form-data, you'll have to build the POST body yourself and set the headers from that. The included urllib3 tools should be able to do this for you:

from requests.packages.urllib3.fields import RequestField
from requests.packages.urllib3.filepost import encode_multipart_formdata

fields = []    
for name, (filename, contents, mimetype) in files.items():
    rf = RequestField(name=name, data=contents,
                      filename=filename)
    rf.make_multipart(content_disposition='attachment', content_type=mimetype)
    fields.append(rf)

post_body, content_type = encode_multipart_formdata(fields)
content_type = ''.join(('multipart/mixed',) + content_type.partition(';')[1:])

headers = {'Content-Type': content_type}
requests.post('http://example.com', data=post_body, headers=headers, params=params)

or you could use the email package to do the same:

from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

body = MIMEMultipart()
for name, (filename, contents, mimetype) in files.items():
    part = MIMEText(contents, _subtype=mimetype.partition('/')[-1], _charset='utf8')
    part.add_header('Content-Disposition', 'attachment', filename=filename)
    body.attach(part)

post_body = body.as_string().partition('

')[-1]
content_type = body['content-type']

headers = {'Content-Type': content_type}
requests.post('http://example.com', data=post_body, headers=headers, params=params)

but take into account that this method expects you to set a character set (I assumed UTF-8 for JSON and XML) and that it'll more than likely use Base64 encoding for the contents:

>>> body = MIMEMultipart()
>>> for name, (filename, contents, mimetype) in files.items():
...     part = MIMEText(contents, _subtype=mimetype.partition('/')[-1], _charset='utf8')
...     part.add_header('Content-Disposition', 'attachment', filename=filename)
...     body.attach(part)
... 
>>> post_body = body.as_string().partition('

')[-1]
>>> content_type = body['content-type']
>>> print post_body
--===============1364782689914852112==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="baz.txt"

YmF6IGNvbnRlbnRz

--===============1364782689914852112==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="bar.txt"

YmFyIGNvbnRlbnRz

--===============1364782689914852112==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="foo.txt"

Zm9vCmNvbnRlbnRzCg==

--===============1364782689914852112==--

>>> print content_type
multipart/mixed; boundary="===============1364782689914852112=="

这篇关于如何使用 python 请求 POST JSON/xml 文件的多部分列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！